-- Logs begin at Sun 2022-10-23 19:27:30 UTC, end at Sun 2022-10-23 20:30:57 UTC. -- Oct 23 19:51:43.800020 dv-cinder-862432-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 23 19:51:43.807289 dv-cinder-862432-1-iscsi devstack@keystone.service[81400]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 23 19:51:43.809296 dv-cinder-862432-1-iscsi devstack@keystone.service[81400]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Oct 23 19:51:43 2022] *** Oct 23 19:51:43.809296 dv-cinder-862432-1-iscsi devstack@keystone.service[81400]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 23 19:51:43.809296 dv-cinder-862432-1-iscsi devstack@keystone.service[81400]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 23 19:51:43.809296 dv-cinder-862432-1-iscsi devstack@keystone.service[81400]: nodename: dv-cinder-862432-1-iscsi Oct 23 19:51:43.809296 dv-cinder-862432-1-iscsi devstack@keystone.service[81400]: machine: x86_64 Oct 23 19:51:43.809296 dv-cinder-862432-1-iscsi devstack@keystone.service[81400]: clock source: unix Oct 23 19:51:43.809296 dv-cinder-862432-1-iscsi devstack@keystone.service[81400]: pcre jit disabled Oct 23 19:51:43.809797 dv-cinder-862432-1-iscsi devstack@keystone.service[81400]: detected number of CPU cores: 4 Oct 23 19:51:43.809797 dv-cinder-862432-1-iscsi devstack@keystone.service[81400]: current working directory: / Oct 23 19:51:43.809797 dv-cinder-862432-1-iscsi devstack@keystone.service[81400]: detected binary path: /usr/bin/uwsgi-core Oct 23 19:51:43.809797 dv-cinder-862432-1-iscsi devstack@keystone.service[81400]: your processes number limit is 47680 Oct 23 19:51:43.809797 dv-cinder-862432-1-iscsi devstack@keystone.service[81400]: your memory page size is 4096 bytes Oct 23 19:51:43.809797 dv-cinder-862432-1-iscsi devstack@keystone.service[81400]: detected max file descriptor number: 2048 Oct 23 19:51:43.809797 dv-cinder-862432-1-iscsi devstack@keystone.service[81400]: lock engine: pthread robust mutexes Oct 23 19:51:43.809797 dv-cinder-862432-1-iscsi devstack@keystone.service[81400]: thunder lock: enabled Oct 23 19:51:43.809797 dv-cinder-862432-1-iscsi devstack@keystone.service[81400]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 23 19:51:43.809797 dv-cinder-862432-1-iscsi devstack@keystone.service[81400]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 23 19:51:43.851030 dv-cinder-862432-1-iscsi devstack@keystone.service[81400]: Python main interpreter initialized at 0x56124fcee9c0 Oct 23 19:51:43.851030 dv-cinder-862432-1-iscsi devstack@keystone.service[81400]: python threads support enabled Oct 23 19:51:43.851030 dv-cinder-862432-1-iscsi devstack@keystone.service[81400]: your server socket listen backlog is limited to 100 connections Oct 23 19:51:43.851030 dv-cinder-862432-1-iscsi devstack@keystone.service[81400]: your mercy for graceful operations on workers is 90 seconds Oct 23 19:51:43.851400 dv-cinder-862432-1-iscsi devstack@keystone.service[81400]: mapped 403077 bytes (393 KB) for 2 cores Oct 23 19:51:43.851502 dv-cinder-862432-1-iscsi devstack@keystone.service[81400]: *** Operational MODE: preforking *** Oct 23 19:51:43.851774 dv-cinder-862432-1-iscsi devstack@keystone.service[81400]: *** uWSGI is running in multiple interpreter mode *** Oct 23 19:51:43.851774 dv-cinder-862432-1-iscsi devstack@keystone.service[81400]: spawned uWSGI master process (pid: 81400) Oct 23 19:51:43.851970 dv-cinder-862432-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 23 19:51:43.853046 dv-cinder-862432-1-iscsi devstack@keystone.service[81400]: spawned uWSGI worker 1 (pid: 81411, cores: 1) Oct 23 19:51:43.853046 dv-cinder-862432-1-iscsi devstack@keystone.service[81400]: spawned uWSGI worker 2 (pid: 81412, cores: 1) Oct 23 19:51:43.853046 dv-cinder-862432-1-iscsi devstack@keystone.service[81400]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 23 19:51:48.104047 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 23 19:51:48.104047 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: from cryptography.utils import int_from_bytes Oct 23 19:51:48.104745 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 23 19:51:48.104745 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: from cryptography.utils import int_from_bytes Oct 23 19:51:48.105183 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 23 19:51:48.105183 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: from cryptography.utils import int_from_bytes Oct 23 19:51:48.105318 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 23 19:51:48.105318 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: from cryptography.utils import int_from_bytes Oct 23 19:51:48.154163 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 23 19:51:48.154383 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 23 19:51:48.154499 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 23 19:51:48.154608 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] command line args: [] {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 23 19:51:48.154691 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 23 19:51:48.154795 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] command line args: [] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 23 19:51:48.154872 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 23 19:51:48.154988 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ================================================================================ {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 23 19:51:48.155076 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 23 19:51:48.155231 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ================================================================================ {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 23 19:51:48.155309 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] admin_token = **** {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.155470 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] config_dir = [] {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.155552 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] admin_token = **** {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.155657 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.155736 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] config_dir = [] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.155837 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] config_source = [] {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.155919 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.156016 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] debug = True {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.156098 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] config_source = [] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.156213 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.156463 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] debug = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.156463 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.156600 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.156732 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.156807 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.156871 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.157026 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.157109 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.157265 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.157354 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.157493 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] list_limit = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.157577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] list_limit = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.157679 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] log_config_append = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.157758 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] log_config_append = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.157851 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.157932 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.157998 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] log_dir = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.158060 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] log_dir = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.158169 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] log_file = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.158302 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] log_file = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.158418 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.158501 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.158663 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.158741 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.158904 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.159021 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.159189 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.159328 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.159328 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.159447 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.159506 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.159570 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.159728 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.159811 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.159959 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.160068 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.160180 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.160273 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.160407 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.160512 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.160631 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.160720 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.160858 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.160943 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.161101 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.161185 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.161347 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.161422 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.161619 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.161704 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.162100 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.162191 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.162300 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] publish_errors = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.162452 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] publish_errors = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.162527 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.162598 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.162749 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.162873 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.162971 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.163038 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.163154 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.163254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.163389 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.163466 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.163619 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] standard_threads = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.163696 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] standard_threads = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.163849 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.163972 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.164089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.164166 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.164249 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.164324 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.164425 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] use_journal = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.164503 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] use_journal = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.164624 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] use_json = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.164702 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] use_json = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.164774 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] use_stderr = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.164843 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] use_stderr = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.164912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] use_syslog = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.164986 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] use_syslog = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.165129 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.165206 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 19:51:48.165381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.165458 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.165618 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.165693 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.165874 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.165950 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.166113 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.166189 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.166383 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.166460 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.166622 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.166706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.166870 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.166946 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.167105 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.167180 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.167353 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.167429 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.167598 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.167672 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.167848 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.167929 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.168079 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.168156 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.168324 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.168405 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.168555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.168628 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.168802 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.168878 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.169046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.169122 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.169283 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.169359 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.169537 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.169614 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.169789 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.169871 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.170034 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.170114 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.170284 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.170358 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.170526 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.170604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.170788 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.170875 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.171038 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.171114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.171284 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.171360 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.171514 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.171597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.171745 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.171827 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.171972 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.172053 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.172204 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.172783 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.172979 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.173055 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.173228 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.173304 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.173460 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.173543 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.173689 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.173764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.173936 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.174011 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.174181 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.174297 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.174443 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.174521 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.174691 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.174768 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.174922 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.175007 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.175158 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.175235 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.175393 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.175469 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.175624 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.175711 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.175853 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.175929 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.176079 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.176161 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.176308 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.176458 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.176533 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.176601 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.176774 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.176852 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.177002 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.177085 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.177232 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.177308 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.177461 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.177537 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.177695 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.177779 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.177925 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.178007 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.178320 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.178445 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.178824 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.178929 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.179023 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.179115 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.179227 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.179300 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.179413 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.179497 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.179574 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.179646 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.179798 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.179882 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.179963 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.180078 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.180157 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.180271 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.180345 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.180458 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.180539 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.180612 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.180731 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.180815 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.180895 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.180975 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.181089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.181168 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.181318 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.181400 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.181553 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.181633 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.181795 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.181912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.182057 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.182142 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.182314 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.182433 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.182561 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.182639 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.182804 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.182883 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.183036 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.183113 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.183269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.183347 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.183502 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.183614 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.183758 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.183840 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.183976 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.184055 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.184209 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.184287 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.184445 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.184522 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.184678 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.184754 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.184918 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.184997 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.185210 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.185304 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.185384 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.185456 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.185602 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.185688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.185833 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.185909 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.186063 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.186139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.186342 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.186423 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.186579 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.186658 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.186822 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.187164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.187311 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.187391 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.187545 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.187623 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.187784 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.187860 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.188592 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.188683 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.188810 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.188894 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.189069 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.189199 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.189305 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.189391 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.189536 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.189613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.189762 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.189838 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.189995 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.190080 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.190221 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.190361 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.190481 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.190559 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.190713 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.190803 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.190962 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.191039 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.191193 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.191271 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.191448 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.191532 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.191706 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.191783 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.191943 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.192093 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.192182 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] auth.external = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.192255 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] auth.external = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.192389 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.192480 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.192682 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.192760 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.192910 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.192992 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.193139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] auth.password = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.193231 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] auth.password = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.193364 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] auth.token = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.193440 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] auth.token = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.193599 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.193686 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.193832 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.193954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.194075 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.194172 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.194329 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.194417 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.194573 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.194670 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.194816 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.194896 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.195041 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.195120 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.195273 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] credential.caching = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.195360 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] credential.caching = True {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.195516 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.195594 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.195761 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.195840 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.196003 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.196091 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.196252 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.196339 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.196493 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.196582 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.196726 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.196813 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.196968 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.197046 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.197216 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.197295 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.197447 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.197534 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.197697 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.197777 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.197976 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.198054 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.198213 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.198335 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.198492 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.198573 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.198730 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.198810 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.198983 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.199072 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.199213 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] federation.caching = True {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.199292 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] federation.caching = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.199445 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.199525 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.199686 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.199774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.199929 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.200009 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.200156 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.200241 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.200399 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.200496 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.200630 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.200709 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.200878 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.200958 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.201110 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.201190 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.201359 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.201440 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.201590 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.201677 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.201832 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.201912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.202062 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] identity.caching = True {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.202145 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] identity.caching = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.202327 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.202412 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.202567 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.202644 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.202797 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.202877 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.203028 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.203106 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.203271 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.203352 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.203501 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.203581 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.203756 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.203834 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.204003 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.204084 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.204248 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.204327 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.204476 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.204554 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.204701 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.204780 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.204927 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.205005 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.205165 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.205244 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.205409 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.205489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.205651 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.205730 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.205904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.205984 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.206145 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.206226 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.206455 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.206534 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.206696 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.206781 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.206933 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.207015 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.207162 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.207242 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.207392 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.207471 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.207620 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.207701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.207852 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.207932 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.208085 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.208164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.208321 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.208401 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.208563 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.208642 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.208791 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.208870 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.209033 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.209114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.209278 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.209369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.209508 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.209587 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.209750 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.209836 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.209991 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.210076 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.210220 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.210329 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.210469 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.210555 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.210696 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.210777 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.210930 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.211018 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.211162 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.211249 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.211515 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.211594 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.211756 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.211848 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.211992 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.212072 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.212240 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.212321 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.212488 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.212576 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.212720 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.212800 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.212950 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.213028 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.213194 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.213281 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.213440 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.213519 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.213670 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.213748 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.213903 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.213991 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.214138 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.214217 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.214386 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.user = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.214557 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.214633 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.214717 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.214867 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.214947 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.215097 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.215176 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.215338 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.215430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.215582 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.215662 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.215822 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.215904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.216053 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.216145 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.216281 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.216360 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.216509 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.216588 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.216741 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.216828 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.216973 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.217052 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.217201 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.217279 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.217444 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.217531 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.217688 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.217768 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.217933 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.218012 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.218172 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.218287 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.218439 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.218520 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.218669 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.218748 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.218910 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.219001 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.219164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.219243 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.219400 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.219479 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.219637 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.219764 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.219872 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.219968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.220113 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.220192 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.220356 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.220444 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.220591 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.220673 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.220841 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.220920 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.221071 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.221159 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.221310 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.221395 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.221547 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.221642 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.221782 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.221870 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.222016 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.222093 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.222272 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.222353 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.222516 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.222606 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.222744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.222828 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.222979 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.223057 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.223216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] resource.caching = True {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.223302 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] resource.caching = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.223464 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.223542 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.223706 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.223784 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.223939 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.224059 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.224184 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.224263 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.224431 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.224508 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.224661 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.224746 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.224904 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.224982 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.225139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.225224 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.225371 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.225524 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.225599 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] role.caching = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.225670 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] role.caching = True {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.225835 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] role.driver = sql {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.225912 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] role.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.226063 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.226141 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.226324 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.226405 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.226571 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.226650 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.226816 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.226895 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.227057 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.227134 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.227304 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.227388 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.227542 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.227620 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.227784 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.227861 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.228025 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.228101 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.228253 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.228332 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.228494 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.228571 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.228737 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.228815 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.228978 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.229058 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.229222 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.229299 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.229613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.229691 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.229845 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.229923 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.230090 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.230168 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.230359 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.230439 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.230605 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.230686 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.230850 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.230927 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.231078 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.231164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.231327 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.231406 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.231574 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.231651 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.231807 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.231891 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.232036 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.232113 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.232261 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.232338 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.232491 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.232576 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.232737 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.232814 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.232989 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.233066 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.233233 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.233318 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.233467 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.233544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.233697 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.233774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.233925 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.234011 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.234158 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] token.caching = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.234260 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] token.caching = True {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.234415 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.234492 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.234662 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.235251 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.235416 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.235494 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.235671 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.235748 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.235914 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.235999 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.236149 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.236226 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.236387 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.236464 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.236626 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.236711 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.236867 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.236945 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.237101 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.237178 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.237331 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.237417 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.237563 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.237642 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.237807 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.237884 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.238078 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.238163 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.238326 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.238421 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.238565 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.238653 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.238823 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.238903 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.239077 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] database.connection = **** {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.239155 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.connection = **** {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.239318 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.239396 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.239559 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.239636 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.239792 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.239870 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.240028 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.240107 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.240256 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.240333 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.240489 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.240566 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.240717 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.240795 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.240949 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.241027 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.241188 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.241265 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.241417 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.241495 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.241656 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.241733 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.241885 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.241962 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.242130 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.242209 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.242380 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.242458 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.242622 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.242701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.242851 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.242929 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.243084 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.243168 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.243316 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.243393 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 19:51:48.243473 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81411) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 23 19:51:48.243545 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81412) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 23 19:51:48.272860 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.273034 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.273118 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.273256 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.273340 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.273479 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.281580 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.282364 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.311084 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.311309 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.311536 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.311769 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.363307 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.363598 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.411499 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.411727 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.412300 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.412538 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.567286 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.567470 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.567645 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.567829 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.567965 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.568095 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.568230 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.568358 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.568498 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.568498 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.568735 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.568874 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.590948 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.591135 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.591291 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.591421 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.591547 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.591686 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.591827 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.591955 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.592081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.592207 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.592341 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.592466 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.592591 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.592716 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.603750 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.603979 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.616938 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.617168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.617298 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.617425 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.617563 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.617690 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.617816 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.617941 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.618067 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.618192 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.618354 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.618486 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81412) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.618627 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.618627 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81411) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 19:51:48.634941 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.635149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 23 19:51:48.635499 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.635615 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.635688 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 23 19:51:48.635752 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 23 19:51:48.635959 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.636064 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.636223 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.636308 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 23 19:51:48.636530 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.636728 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.636820 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.636989 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.637295 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.637564 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.646375 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.646655 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.646926 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.647011 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.647210 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.647295 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.647526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.647814 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.654138 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.654853 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.657576 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.657857 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.658138 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.658285 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.658444 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.658575 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.658723 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.658861 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.658981 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.659148 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.659230 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.659434 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.659515 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.659699 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.659874 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.659956 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.660151 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.660232 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.660420 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.660610 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.660892 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.661172 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.683307 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.684207 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.684770 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.685055 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.685679 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.685965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.692567 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.692870 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.693127 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.696538 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.696538 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.696538 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.699806 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.700098 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.702763 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.702984 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.704754 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.705593 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.705593 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.705929 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.705929 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.706191 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.706522 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.706875 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.707284 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.707588 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.707874 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.708164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.708754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.708754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.709142 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.709364 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.724282 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.724570 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.726096 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.726407 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.728981 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.730774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.732679 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.732977 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.734018 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.734350 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.737794 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.739180 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.740961 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.742362 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.744207 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.744510 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.744800 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.745087 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.745377 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.745597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.745724 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.745903 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.746197 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.746514 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.746815 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.747097 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.761375 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.761666 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.761935 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.762202 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.762970 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.763264 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.763539 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.763812 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.769574 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.771143 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.771283 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.772688 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.772859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.773034 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.774349 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.774697 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.775686 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.775972 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.776369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.776654 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.776937 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.777357 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.777648 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.778042 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.778359 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.778658 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.788555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.788841 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.789220 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.789515 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.789809 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.790110 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.790365 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.790497 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.790669 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.791039 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.791342 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.791642 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.791936 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.792222 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.805675 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.807618 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.808587 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.810563 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.811484 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.813195 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.813873 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.814839 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.815134 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.815369 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.815452 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.816990 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.817328 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.817573 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.821694 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.823182 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.823940 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.825416 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.826176 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.826494 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.826780 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.827054 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.828454 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.829260 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.829548 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.829826 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.835366 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.835658 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.835936 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.837647 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.837940 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.838223 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.843690 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81412) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.843973 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.844811 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.845090 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.845360 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.845630 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.845899 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.846182 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.846482 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.846625 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81411) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 19:51:48.846749 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.846916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.847021 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.847202 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.847342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.847484 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.847609 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.847758 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.847869 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81412) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.848032 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.848309 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.848600 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.848889 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.849167 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.849441 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.849760 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.850031 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.850318 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81411) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 19:51:48.880654 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56124fcee9c0 pid: 81412 (default app) Oct 23 19:51:48.883156 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56124fcee9c0 pid: 81411 (default app) Oct 23 19:51:48.883967 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98413af1-50a6-482d-86a8-5f98a90e2995 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:51:48.884256 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98413af1-50a6-482d-86a8-5f98a90e2995 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:51:48.884527 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98413af1-50a6-482d-86a8-5f98a90e2995 None None] PATH_INFO: `/v3` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:51:48.888838 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1/1] 10.222.0.13 () {50 vars in 791 bytes} [Sun Oct 23 19:51:48 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:51:55.586522 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58295ad-d866-4179-b8b1-2afab83f1451 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:51:55.587030 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58295ad-d866-4179-b8b1-2afab83f1451 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:51:55.587112 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58295ad-d866-4179-b8b1-2afab83f1451 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:51:55.591511 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1/2] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:51:55 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:51:55.597189 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5154d-736c-4907-8dd3-519c1636d704 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:51:55.597751 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5154d-736c-4907-8dd3-519c1636d704 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:51:55.597751 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb5154d-736c-4907-8dd3-519c1636d704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:51:55.667485 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: INFO dbcounter [None req-8cb5154d-736c-4907-8dd3-519c1636d704 None None] Registered counter for database keystone Oct 23 19:51:55.686848 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG oslo_db.sqlalchemy.engines [None req-8cb5154d-736c-4907-8dd3-519c1636d704 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81412) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 23 19:51:55.693804 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writer thread running {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 23 19:51:55.870346 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG passlib.handlers.bcrypt [None req-8cb5154d-736c-4907-8dd3-519c1636d704 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81412) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 23 19:51:55.870658 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG passlib.handlers.bcrypt [None req-8cb5154d-736c-4907-8dd3-519c1636d704 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81412) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 23 19:51:55.928049 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-8cb5154d-736c-4907-8dd3-519c1636d704 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:51:55.929667 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-8cb5154d-736c-4907-8dd3-519c1636d704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:51:56.057492 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 2/3] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:51:55 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 461 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 23 19:51:56.527990 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58295ad-d866-4179-b8b1-2afab83f1451 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:51:56.597595 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: INFO dbcounter [None req-a58295ad-d866-4179-b8b1-2afab83f1451 None None] Registered counter for database keystone Oct 23 19:51:56.618525 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG oslo_db.sqlalchemy.engines [None req-a58295ad-d866-4179-b8b1-2afab83f1451 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81411) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 23 19:51:56.624863 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writer thread running {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 23 19:51:56.744092 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58295ad-d866-4179-b8b1-2afab83f1451 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:51:56.748357 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f43528-7ed5-4b77-81b3-d9b4d4b86498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:51:56.748979 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f43528-7ed5-4b77-81b3-d9b4d4b86498 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:51:56.749261 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f43528-7ed5-4b77-81b3-d9b4d4b86498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:51:56.749534 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f43528-7ed5-4b77-81b3-d9b4d4b86498 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:51:56.758193 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f43528-7ed5-4b77-81b3-d9b4d4b86498 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:51:56.809122 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: WARNING keystone.common.rbac_enforcer.enforcer [None req-96f43528-7ed5-4b77-81b3-d9b4d4b86498 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 23 19:51:56.809413 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f43528-7ed5-4b77-81b3-d9b4d4b86498 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:51:56.819448 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 2/4] 10.222.0.13 () {58 vars in 1324 bytes} [Sun Oct 23 19:51:56 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 19:51:56.825374 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb5154d-736c-4907-8dd3-519c1636d704 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:51:56.842197 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb5154d-736c-4907-8dd3-519c1636d704 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:51:56.847152 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f71f2f-81a8-4ea4-acc9-9e883c149e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:51:56.848246 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f71f2f-81a8-4ea4-acc9-9e883c149e98 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:51:56.848527 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f71f2f-81a8-4ea4-acc9-9e883c149e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:51:56.848796 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f71f2f-81a8-4ea4-acc9-9e883c149e98 admin admin] PATH_INFO: `/v3/projects/c6d0fd21d5e64f76b7baefae60befbed` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:51:56.852568 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f71f2f-81a8-4ea4-acc9-9e883c149e98 admin admin] RBAC: Authorizing `identity:get_project(project_id=c6d0fd21d5e64f76b7baefae60befbed)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:51:56.902445 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: WARNING keystone.common.rbac_enforcer.enforcer [None req-16f71f2f-81a8-4ea4-acc9-9e883c149e98 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 23 19:51:56.902789 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f71f2f-81a8-4ea4-acc9-9e883c149e98 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:51:56.907850 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 3/5] 10.222.0.13 () {56 vars in 1207 bytes} [Sun Oct 23 19:51:56 2022] GET /identity/v3/projects/c6d0fd21d5e64f76b7baefae60befbed => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:51:59.010355 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e940464-c472-48c9-bde7-be41d909d52a None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:51:59.010974 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e940464-c472-48c9-bde7-be41d909d52a None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:51:59.010974 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e940464-c472-48c9-bde7-be41d909d52a None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:51:59.015062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 3/6] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:51:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:51:59.021133 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddce04d-f802-41df-b83f-f5ee4b824550 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:51:59.021133 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddce04d-f802-41df-b83f-f5ee4b824550 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:51:59.021560 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddce04d-f802-41df-b83f-f5ee4b824550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:51:59.041363 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-1ddce04d-f802-41df-b83f-f5ee4b824550 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:51:59.042888 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-1ddce04d-f802-41df-b83f-f5ee4b824550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:51:59.051401 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 4/7] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:51:59 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 23 19:51:59.317931 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e940464-c472-48c9-bde7-be41d909d52a None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:51:59.329982 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e940464-c472-48c9-bde7-be41d909d52a None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:51:59.334146 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7dae94-fa42-4372-bea7-151d8f3b94cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:51:59.334776 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7dae94-fa42-4372-bea7-151d8f3b94cf admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:51:59.335059 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7dae94-fa42-4372-bea7-151d8f3b94cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:51:59.335333 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7dae94-fa42-4372-bea7-151d8f3b94cf admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:51:59.342967 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7dae94-fa42-4372-bea7-151d8f3b94cf admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:51:59.344752 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7dae94-fa42-4372-bea7-151d8f3b94cf admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:51:59.353433 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 4/8] 10.222.0.13 () {58 vars in 1324 bytes} [Sun Oct 23 19:51:59 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 23 19:51:59.358679 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddce04d-f802-41df-b83f-f5ee4b824550 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:51:59.368567 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddce04d-f802-41df-b83f-f5ee4b824550 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:51:59.372924 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ed6c75-46f0-4443-b7d9-79aea3e72670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:51:59.373546 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed6c75-46f0-4443-b7d9-79aea3e72670 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:51:59.373828 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed6c75-46f0-4443-b7d9-79aea3e72670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:51:59.374107 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ed6c75-46f0-4443-b7d9-79aea3e72670 admin admin] PATH_INFO: `/v3/users/55e5de29fcfb4f0abcb119685810b1da` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:51:59.376040 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed6c75-46f0-4443-b7d9-79aea3e72670 admin admin] RBAC: Authorizing `identity:get_user(user_id=55e5de29fcfb4f0abcb119685810b1da)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:51:59.377850 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ed6c75-46f0-4443-b7d9-79aea3e72670 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:51:59.379417 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 5/9] 10.222.0.13 () {56 vars in 1198 bytes} [Sun Oct 23 19:51:59 2022] GET /identity/v3/users/55e5de29fcfb4f0abcb119685810b1da => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:01.536262 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d38125-b6ba-4400-bca8-f02e826c20c4 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:01.536794 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d38125-b6ba-4400-bca8-f02e826c20c4 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:01.536794 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d38125-b6ba-4400-bca8-f02e826c20c4 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:01.538126 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 5/10] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:01.543135 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef9ffa-543d-4718-ad31-8b05c91b8ec3 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:01.543516 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef9ffa-543d-4718-ad31-8b05c91b8ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:01.543686 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfef9ffa-543d-4718-ad31-8b05c91b8ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:01.561837 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-bfef9ffa-543d-4718-ad31-8b05c91b8ec3 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:01.563360 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-bfef9ffa-543d-4718-ad31-8b05c91b8ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:01.585660 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820a2da-96cd-47c1-9324-fecb956c4cba None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:01.586162 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820a2da-96cd-47c1-9324-fecb956c4cba None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:01.586471 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820a2da-96cd-47c1-9324-fecb956c4cba None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:01.587793 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 6/11] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 19:52:01.592938 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4b94b-c56a-4d8b-9227-082013e27761 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:01.593217 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4b94b-c56a-4d8b-9227-082013e27761 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:01.593490 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe4b94b-c56a-4d8b-9227-082013e27761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:01.634724 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG passlib.handlers.bcrypt [None req-1fe4b94b-c56a-4d8b-9227-082013e27761 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81411) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 23 19:52:01.635107 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG passlib.handlers.bcrypt [None req-1fe4b94b-c56a-4d8b-9227-082013e27761 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81411) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 23 19:52:01.645788 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 6/12] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:01.684779 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-1fe4b94b-c56a-4d8b-9227-082013e27761 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:01.686385 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-1fe4b94b-c56a-4d8b-9227-082013e27761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:01.784599 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 7/13] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 19:52:01.916605 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef9ffa-543d-4718-ad31-8b05c91b8ec3 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:01.940711 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:02.024005 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfef9ffa-543d-4718-ad31-8b05c91b8ec3 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:02.052246 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4b94b-c56a-4d8b-9227-082013e27761 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:02.057312 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54457e8-ee52-46a0-87dd-e47675a5a84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:02.057957 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54457e8-ee52-46a0-87dd-e47675a5a84a None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:02.058267 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54457e8-ee52-46a0-87dd-e47675a5a84a None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:02.058558 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54457e8-ee52-46a0-87dd-e47675a5a84a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:02.070860 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54457e8-ee52-46a0-87dd-e47675a5a84a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:02.080330 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-b54457e8-ee52-46a0-87dd-e47675a5a84a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:52:02.080330 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:02.080330 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:02.080330 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:02.080330 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:02.080330 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:02.080330 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:02.080330 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:02.080330 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:02.080330 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:02.080330 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:02.080330 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:02.080330 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:02.080330 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:02.080330 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:02.080330 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:02.081256 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:02.081256 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:02.081256 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:02.081256 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:02.081256 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:02.081256 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:02.081256 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:02.081256 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:02.081256 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:02.081256 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:02.081256 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:02.081256 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:02.081256 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:02.081256 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:02.081256 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:02.081256 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:02.082096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:02.082096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:02.082096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:02.082096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:02.082096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:02.082096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:02.082096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:02.082096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:02.082096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:02.082096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:02.082096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:52:02.082096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:52:02.082096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 7/14] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Oct 23 19:52:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:52:02.088599 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54457e8-ee52-46a0-87dd-e47675a5a84a None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:02.157605 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe4b94b-c56a-4d8b-9227-082013e27761 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:02.159909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:02.168409 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:02.187016 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54457e8-ee52-46a0-87dd-e47675a5a84a None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:02.195514 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ea9f05-b50c-4122-956d-5c1d9112dce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:02.196386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea9f05-b50c-4122-956d-5c1d9112dce5 None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:02.196668 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea9f05-b50c-4122-956d-5c1d9112dce5 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:02.196942 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ea9f05-b50c-4122-956d-5c1d9112dce5 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:02.197897 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ea9f05-b50c-4122-956d-5c1d9112dce5 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:02.199345 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ea9f05-b50c-4122-956d-5c1d9112dce5 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:02.211112 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.sql.core [None req-38ea9f05-b50c-4122-956d-5c1d9112dce5 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 23 19:52:02.211112 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 23 19:52:02.211112 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [parameters: {'id': '4923fa3a86be4f6e878d657b23e02809', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 23 19:52:02.211112 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81411) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 23 19:52:02.222574 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ffc67-e9fc-4f93-9e22-4a95dac7d3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:02.223178 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ffc67-e9fc-4f93-9e22-4a95dac7d3e1 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:02.223456 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ffc67-e9fc-4f93-9e22-4a95dac7d3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:02.223729 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49ffc67-e9fc-4f93-9e22-4a95dac7d3e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:02.224396 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49ffc67-e9fc-4f93-9e22-4a95dac7d3e1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:02.224704 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49ffc67-e9fc-4f93-9e22-4a95dac7d3e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:02.226076 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49ffc67-e9fc-4f93-9e22-4a95dac7d3e1 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:02.229596 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-38ea9f05-b50c-4122-956d-5c1d9112dce5 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 23 19:52:02.229596 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:02.229596 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 23 19:52:02.229596 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 23 19:52:02.229596 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 23 19:52:02.229596 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 23 19:52:02.229596 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 23 19:52:02.229596 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application result = self._query(query) Oct 23 19:52:02.229596 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 23 19:52:02.229596 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application conn.query(q) Oct 23 19:52:02.229596 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 23 19:52:02.229596 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 23 19:52:02.229596 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 23 19:52:02.229596 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application result.read() Oct 23 19:52:02.230597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 23 19:52:02.230597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 23 19:52:02.230597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 23 19:52:02.230597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 23 19:52:02.230597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 23 19:52:02.230597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 23 19:52:02.230597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 23 19:52:02.230597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 23 19:52:02.230597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 23 19:52:02.230597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:52:02.230597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 23 19:52:02.230597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:52:02.230597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:02.230597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 23 19:52:02.230597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 23 19:52:02.230597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 23 19:52:02.230597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 23 19:52:02.230597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 23 19:52:02.231526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application next(self.gen) Oct 23 19:52:02.231526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 23 19:52:02.231526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application yield resource Oct 23 19:52:02.231526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 23 19:52:02.231526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application next(self.gen) Oct 23 19:52:02.231526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 23 19:52:02.231526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self.session.rollback() Oct 23 19:52:02.231526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 23 19:52:02.231526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self.force_reraise() Oct 23 19:52:02.231526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 23 19:52:02.231526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise self.value Oct 23 19:52:02.231526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 23 19:52:02.231526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 23 19:52:02.231526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 23 19:52:02.231526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application session.commit() Oct 23 19:52:02.231526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 23 19:52:02.231526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 23 19:52:02.232375 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 23 19:52:02.232375 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._prepare_impl() Oct 23 19:52:02.232375 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 23 19:52:02.232375 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self.session.flush() Oct 23 19:52:02.232375 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 23 19:52:02.232375 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._flush(objects) Oct 23 19:52:02.232375 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 23 19:52:02.232375 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 23 19:52:02.232375 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 23 19:52:02.232375 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application compat.raise_( Oct 23 19:52:02.232375 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 23 19:52:02.232375 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception Oct 23 19:52:02.232375 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 23 19:52:02.232375 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application flush_context.execute() Oct 23 19:52:02.232375 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 23 19:52:02.232375 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rec.execute(self) Oct 23 19:52:02.232375 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 23 19:52:02.233270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 23 19:52:02.233270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 23 19:52:02.233270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 23 19:52:02.233270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 23 19:52:02.233270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 23 19:52:02.233270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 23 19:52:02.233270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 23 19:52:02.233270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 23 19:52:02.233270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 23 19:52:02.233270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 23 19:52:02.233270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 23 19:52:02.233270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 23 19:52:02.233270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 23 19:52:02.233270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 23 19:52:02.233270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 23 19:52:02.234018 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 23 19:52:02.234018 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception Oct 23 19:52:02.234018 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 23 19:52:02.234018 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 23 19:52:02.234018 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 23 19:52:02.234018 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 23 19:52:02.234018 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 23 19:52:02.234018 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application result = self._query(query) Oct 23 19:52:02.234018 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 23 19:52:02.234018 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application conn.query(q) Oct 23 19:52:02.234018 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 23 19:52:02.234018 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 23 19:52:02.234018 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 23 19:52:02.234018 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application result.read() Oct 23 19:52:02.234018 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 23 19:52:02.234018 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 23 19:52:02.234018 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 23 19:52:02.234872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 23 19:52:02.234872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 23 19:52:02.234872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 23 19:52:02.234872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 23 19:52:02.234872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 23 19:52:02.234872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 23 19:52:02.234872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 23 19:52:02.234872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [parameters: {'id': '4923fa3a86be4f6e878d657b23e02809', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 23 19:52:02.234872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 23 19:52:02.234872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:52:02.234872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 19:52:02.234872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:52:02.234872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:02.234872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 23 19:52:02.234872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 23 19:52:02.236032 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 23 19:52:02.236032 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 23 19:52:02.236032 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 23 19:52:02.236032 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:52:02.236032 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 19:52:02.236032 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:52:02.236032 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:02.236032 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:02.236032 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:02.236032 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:02.236032 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:02.236032 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:02.236032 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:02.236032 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:02.236032 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:02.236032 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:02.236032 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:02.236904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 23 19:52:02.236904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 23 19:52:02.236904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:02.236904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:02.236904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 23 19:52:02.236904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 23 19:52:02.236904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:02.236904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:02.236904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 23 19:52:02.236904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 23 19:52:02.236904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 23 19:52:02.236904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:52:02.236904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 8/15] 10.222.0.13 () {60 vars in 1137 bytes} [Sun Oct 23 19:52:02 2022] POST /identity/v3/domains => generated 190 bytes in 179 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:52:02.236904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ea9f05-b50c-4122-956d-5c1d9112dce5 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:02.239054 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 8/16] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:52:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:02.244330 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ffc67-e9fc-4f93-9e22-4a95dac7d3e1 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:02.337325 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ea9f05-b50c-4122-956d-5c1d9112dce5 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:02.342703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49ffc67-e9fc-4f93-9e22-4a95dac7d3e1 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:02.370758 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d4257b-2eca-4bd0-8423-961d7be059a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:02.371605 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4257b-2eca-4bd0-8423-961d7be059a9 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:02.371891 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4257b-2eca-4bd0-8423-961d7be059a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:02.372169 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d4257b-2eca-4bd0-8423-961d7be059a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:02.377113 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4783005-006d-4dfb-9369-5de2b16a818b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:02.377737 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4783005-006d-4dfb-9369-5de2b16a818b None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:02.378020 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4783005-006d-4dfb-9369-5de2b16a818b None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:02.378328 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4783005-006d-4dfb-9369-5de2b16a818b None admin] PATH_INFO: `/v3/users/55e5de29fcfb4f0abcb119685810b1da` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:02.379482 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4783005-006d-4dfb-9369-5de2b16a818b None admin] RBAC: Authorizing `identity:get_user(user_id=55e5de29fcfb4f0abcb119685810b1da)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:02.380929 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4783005-006d-4dfb-9369-5de2b16a818b None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:02.382555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 9/17] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:52:02 2022] GET /identity/v3/users/55e5de29fcfb4f0abcb119685810b1da => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:02.387881 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4783005-006d-4dfb-9369-5de2b16a818b None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:02.390481 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d4257b-2eca-4bd0-8423-961d7be059a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:02.391972 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d4257b-2eca-4bd0-8423-961d7be059a9 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:02.402627 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:02.407909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-e2d4257b-2eca-4bd0-8423-961d7be059a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:52:02.407909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:02.407909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:02.407909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:02.407909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:02.407909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:02.407909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:02.407909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:02.407909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:02.407909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:02.407909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:02.407909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:02.407909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:52:02.407909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:52:02.407909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:52:02.407909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:52:02.408952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:02.408952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:02.408952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:02.408952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:02.408952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:02.408952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:02.408952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:02.408952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:02.408952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:02.408952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:02.408952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:02.408952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:02.408952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:02.408952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:02.408952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:02.408952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:02.408952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:52:02.409758 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:52:02.409758 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:52:02.409758 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:52:02.409758 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 9/18] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:52:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:52:02.414796 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d4257b-2eca-4bd0-8423-961d7be059a9 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:02.440248 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:02.489409 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4783005-006d-4dfb-9369-5de2b16a818b None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:02.518540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d4257b-2eca-4bd0-8423-961d7be059a9 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:02.522418 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d5252-1155-4621-aba6-c52b5c6ffce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:02.523044 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d5252-1155-4621-aba6-c52b5c6ffce1 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:02.523326 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d5252-1155-4621-aba6-c52b5c6ffce1 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:02.523593 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39d5252-1155-4621-aba6-c52b5c6ffce1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:02.524725 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39d5252-1155-4621-aba6-c52b5c6ffce1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:02.526170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39d5252-1155-4621-aba6-c52b5c6ffce1 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:02.527897 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 10/19] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:52:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:02.533326 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d5252-1155-4621-aba6-c52b5c6ffce1 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:02.551658 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273a07e7-17c8-49cf-83b0-e231ac8e5163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:02.552260 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a07e7-17c8-49cf-83b0-e231ac8e5163 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:02.552540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a07e7-17c8-49cf-83b0-e231ac8e5163 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:02.552812 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273a07e7-17c8-49cf-83b0-e231ac8e5163 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:02.553446 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a07e7-17c8-49cf-83b0-e231ac8e5163 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:02.553746 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a07e7-17c8-49cf-83b0-e231ac8e5163 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:02.555150 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273a07e7-17c8-49cf-83b0-e231ac8e5163 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:02.572487 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 10/20] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:52:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:02.614507 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:02.635413 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39d5252-1155-4621-aba6-c52b5c6ffce1 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:02.671321 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd7add9-6f72-4f82-9a19-07eb376f49fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:02.672005 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7add9-6f72-4f82-9a19-07eb376f49fb None admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:02.672955 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7add9-6f72-4f82-9a19-07eb376f49fb None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:02.673232 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd7add9-6f72-4f82-9a19-07eb376f49fb None admin] PATH_INFO: `/v3/domains/default/users/55e5de29fcfb4f0abcb119685810b1da/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:02.675283 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd7add9-6f72-4f82-9a19-07eb376f49fb None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=55e5de29fcfb4f0abcb119685810b1da, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:02.676751 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd7add9-6f72-4f82-9a19-07eb376f49fb None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:02.793474 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 11/21] 10.222.0.13 () {58 vars in 1361 bytes} [Sun Oct 23 19:52:02 2022] PUT /identity/v3/domains/default/users/55e5de29fcfb4f0abcb119685810b1da/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 19:52:05.101650 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26decc2-7489-409f-a9aa-a96bebc5c4ba None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:05.102145 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26decc2-7489-409f-a9aa-a96bebc5c4ba None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:05.102216 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26decc2-7489-409f-a9aa-a96bebc5c4ba None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:05.103937 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 11/22] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:05.109244 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771fbf2-d6f4-45e6-90f4-3cb00f458c53 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:05.109717 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771fbf2-d6f4-45e6-90f4-3cb00f458c53 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:05.109896 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771fbf2-d6f4-45e6-90f4-3cb00f458c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:05.135180 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-1771fbf2-d6f4-45e6-90f4-3cb00f458c53 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:05.136697 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-1771fbf2-d6f4-45e6-90f4-3cb00f458c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:05.226706 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 12/23] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:05.357235 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8208c46-d290-41bc-a7b7-8ff616918331 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:05.357865 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8208c46-d290-41bc-a7b7-8ff616918331 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:05.358370 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8208c46-d290-41bc-a7b7-8ff616918331 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:05.360782 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 12/24] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:05.365784 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff4202-5469-4c52-bd78-a9bbec09a0c8 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:05.366415 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff4202-5469-4c52-bd78-a9bbec09a0c8 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:05.366928 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ff4202-5469-4c52-bd78-a9bbec09a0c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:05.387496 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-c9ff4202-5469-4c52-bd78-a9bbec09a0c8 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:05.389025 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-c9ff4202-5469-4c52-bd78-a9bbec09a0c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:05.474897 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:05.480207 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 13/25] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:05.608310 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e461d0-463f-49e6-a7a8-ee276ca0eeb5 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:05.608310 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e461d0-463f-49e6-a7a8-ee276ca0eeb5 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:05.608310 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e461d0-463f-49e6-a7a8-ee276ca0eeb5 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:05.609042 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 13/26] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:05.615527 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33877e-c430-4bea-b1e7-20619f3fe93e None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:05.616159 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33877e-c430-4bea-b1e7-20619f3fe93e None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:05.616658 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a33877e-c430-4bea-b1e7-20619f3fe93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:05.634226 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e461d0-463f-49e6-a7a8-ee276ca0eeb5 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:05.647074 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-9a33877e-c430-4bea-b1e7-20619f3fe93e None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:05.650898 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-9a33877e-c430-4bea-b1e7-20619f3fe93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:05.755579 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 14/27] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:05.771203 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bde36d-0c5a-42fa-915a-c02806c2afd0 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:05.771782 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bde36d-0c5a-42fa-915a-c02806c2afd0 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:05.772261 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bde36d-0c5a-42fa-915a-c02806c2afd0 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:05.776793 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 15/28] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 19:52:05.782563 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e729f5a1-cc6a-457c-95ad-fee77af37985 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:05.783089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e729f5a1-cc6a-457c-95ad-fee77af37985 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:05.783584 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e729f5a1-cc6a-457c-95ad-fee77af37985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:05.815931 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-e729f5a1-cc6a-457c-95ad-fee77af37985 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:05.817117 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-e729f5a1-cc6a-457c-95ad-fee77af37985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:05.840174 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:05.892470 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e461d0-463f-49e6-a7a8-ee276ca0eeb5 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:05.934788 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 16/29] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 19:52:05.976164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc70de0-b223-4517-adb6-70b0079892f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:05.976822 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc70de0-b223-4517-adb6-70b0079892f0 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:05.977112 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc70de0-b223-4517-adb6-70b0079892f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:05.977396 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc70de0-b223-4517-adb6-70b0079892f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:05.977505 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e729f5a1-cc6a-457c-95ad-fee77af37985 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:06.014601 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc70de0-b223-4517-adb6-70b0079892f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:06.021840 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-cbc70de0-b223-4517-adb6-70b0079892f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:52:06.021840 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:06.021840 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:06.021840 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:06.021840 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:06.021840 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:06.021840 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:06.021840 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:06.021840 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:06.021840 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:06.021840 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:06.021840 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:06.021840 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:06.021840 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:06.021840 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:06.021840 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:06.026898 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:06.026898 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:06.026898 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:06.026898 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:06.026898 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:06.026898 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:06.026898 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:06.026898 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:06.026898 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:06.026898 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:06.026898 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:06.026898 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:06.026898 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:06.026898 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:06.026898 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:06.026898 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:06.027638 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:06.027638 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:06.027638 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:06.027638 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:06.027638 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:06.027638 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:06.027638 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:06.027638 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:06.027638 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:06.027638 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:06.027638 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:52:06.027638 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:52:06.031170 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 14/30] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Oct 23 19:52:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:52:06.036953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc70de0-b223-4517-adb6-70b0079892f0 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:06.082614 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:06.121599 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e729f5a1-cc6a-457c-95ad-fee77af37985 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:06.157246 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56076196-535b-4ab8-add8-37cf24d81e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:06.157870 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56076196-535b-4ab8-add8-37cf24d81e79 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:06.158169 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56076196-535b-4ab8-add8-37cf24d81e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:06.158474 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56076196-535b-4ab8-add8-37cf24d81e79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:06.169353 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56076196-535b-4ab8-add8-37cf24d81e79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:06.171104 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56076196-535b-4ab8-add8-37cf24d81e79 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:06.189752 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-56076196-535b-4ab8-add8-37cf24d81e79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:52:06.189752 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:06.189752 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:06.189752 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:06.189752 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:06.189752 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:06.189752 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:06.189752 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:06.189752 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:06.189752 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:06.189752 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:06.189752 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:06.189752 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:52:06.189752 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:52:06.189752 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:52:06.189752 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:52:06.190763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:06.190763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:06.190763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:06.190763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:06.190763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:06.190763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:06.190763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:06.190763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:06.190763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:06.190763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:06.190763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:06.190763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:06.190763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:06.190763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:06.190763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:06.190763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:06.190763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:52:06.191537 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:52:06.191537 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:52:06.191537 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:52:06.191537 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 17/31] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:52:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:52:06.191748 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56076196-535b-4ab8-add8-37cf24d81e79 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:06.284323 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc70de0-b223-4517-adb6-70b0079892f0 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:06.295101 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56076196-535b-4ab8-add8-37cf24d81e79 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:06.310965 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:06.326853 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da482aec-a853-45d6-b857-15df8943bbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:06.327552 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da482aec-a853-45d6-b857-15df8943bbce None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:06.327753 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da482aec-a853-45d6-b857-15df8943bbce None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:06.328057 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da482aec-a853-45d6-b857-15df8943bbce None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:06.329287 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da482aec-a853-45d6-b857-15df8943bbce None admin] RBAC: Adding query filter params (name=admin) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:06.329597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da482aec-a853-45d6-b857-15df8943bbce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:06.331323 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da482aec-a853-45d6-b857-15df8943bbce None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:06.336411 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7606c4bf-fdd1-402d-bcc8-abd38b547de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:06.337269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606c4bf-fdd1-402d-bcc8-abd38b547de9 None admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:06.337776 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606c4bf-fdd1-402d-bcc8-abd38b547de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:06.338274 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606c4bf-fdd1-402d-bcc8-abd38b547de9 None admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:06.339148 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7606c4bf-fdd1-402d-bcc8-abd38b547de9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:06.340836 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7606c4bf-fdd1-402d-bcc8-abd38b547de9 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:06.343870 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 15/32] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:52:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:06.344955 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da482aec-a853-45d6-b857-15df8943bbce None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:06.348941 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:06.360235 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 18/33] 10.222.0.13 () {60 vars in 1131 bytes} [Sun Oct 23 19:52:06 2022] POST /identity/v3/roles => generated 225 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:52:06.366412 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7606c4bf-fdd1-402d-bcc8-abd38b547de9 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:06.489303 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7606c4bf-fdd1-402d-bcc8-abd38b547de9 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:06.538992 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c648aa-83cf-449e-83f7-60f842323791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:06.540038 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c648aa-83cf-449e-83f7-60f842323791 None admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:06.540557 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c648aa-83cf-449e-83f7-60f842323791 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:06.541031 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c648aa-83cf-449e-83f7-60f842323791 None admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:06.541926 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c648aa-83cf-449e-83f7-60f842323791 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:06.542590 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da482aec-a853-45d6-b857-15df8943bbce None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:06.546145 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c648aa-83cf-449e-83f7-60f842323791 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:06.577853 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 19/34] 10.222.0.13 () {60 vars in 1131 bytes} [Sun Oct 23 19:52:06 2022] POST /identity/v3/roles => generated 219 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:52:06.579236 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c648aa-83cf-449e-83f7-60f842323791 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:06.627433 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e970a4-0e37-4405-a06f-c2498308a980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:06.628431 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e970a4-0e37-4405-a06f-c2498308a980 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:06.628933 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e970a4-0e37-4405-a06f-c2498308a980 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:06.629529 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e970a4-0e37-4405-a06f-c2498308a980 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:06.630575 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e970a4-0e37-4405-a06f-c2498308a980 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:06.631139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e970a4-0e37-4405-a06f-c2498308a980 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:06.633453 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e970a4-0e37-4405-a06f-c2498308a980 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:06.650979 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 16/35] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:52:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:06.656996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:06.657956 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e970a4-0e37-4405-a06f-c2498308a980 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:06.753805 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c648aa-83cf-449e-83f7-60f842323791 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:06.782664 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:06.814969 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e970a4-0e37-4405-a06f-c2498308a980 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:06.863559 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f834f33-f6fd-4ee4-ab2e-babe8091dd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:06.864201 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f834f33-f6fd-4ee4-ab2e-babe8091dd3a None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:06.864482 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f834f33-f6fd-4ee4-ab2e-babe8091dd3a None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:06.864753 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f834f33-f6fd-4ee4-ab2e-babe8091dd3a None admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:06.865424 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f834f33-f6fd-4ee4-ab2e-babe8091dd3a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:06.866911 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f834f33-f6fd-4ee4-ab2e-babe8091dd3a None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:06.916447 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb8f293-0592-48fb-8928-7ea6afa2a7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:06.917094 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8f293-0592-48fb-8928-7ea6afa2a7a2 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:06.917382 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8f293-0592-48fb-8928-7ea6afa2a7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:06.917662 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb8f293-0592-48fb-8928-7ea6afa2a7a2 None admin] PATH_INFO: `/v3/users/55e5de29fcfb4f0abcb119685810b1da` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:06.918846 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb8f293-0592-48fb-8928-7ea6afa2a7a2 None admin] RBAC: Authorizing `identity:get_user(user_id=55e5de29fcfb4f0abcb119685810b1da)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:06.920317 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb8f293-0592-48fb-8928-7ea6afa2a7a2 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:06.922598 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 20/36] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:52:06 2022] GET /identity/v3/users/55e5de29fcfb4f0abcb119685810b1da => generated 247 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:06.923901 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef4a18-b8fe-4f2a-bc00-83896f8702ee None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:06.924195 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef4a18-b8fe-4f2a-bc00-83896f8702ee None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:06.924475 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef4a18-b8fe-4f2a-bc00-83896f8702ee None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:06.925717 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 21/37] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:06.930664 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef4a18-b8fe-4f2a-bc00-83896f8702ee None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:07.006325 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 17/38] 10.222.0.13 () {60 vars in 1141 bytes} [Sun Oct 23 19:52:06 2022] POST /identity/v3/projects => generated 301 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:52:07.007706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73374a25-2377-47d5-9835-7f817a1ae5fa None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:07.008000 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73374a25-2377-47d5-9835-7f817a1ae5fa None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:07.008282 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73374a25-2377-47d5-9835-7f817a1ae5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:07.032399 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-73374a25-2377-47d5-9835-7f817a1ae5fa None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:07.034040 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-73374a25-2377-47d5-9835-7f817a1ae5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:07.059713 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef4a18-b8fe-4f2a-bc00-83896f8702ee None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:07.079481 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:07.111119 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb01cd2-399d-45ef-9f8d-cddcac94e844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:07.111779 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb01cd2-399d-45ef-9f8d-cddcac94e844 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:07.112061 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb01cd2-399d-45ef-9f8d-cddcac94e844 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:07.112352 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb01cd2-399d-45ef-9f8d-cddcac94e844 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:07.113567 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb01cd2-399d-45ef-9f8d-cddcac94e844 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:07.115151 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb01cd2-399d-45ef-9f8d-cddcac94e844 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:07.117088 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 22/39] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:52:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:07.123101 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb01cd2-399d-45ef-9f8d-cddcac94e844 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:07.138391 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 18/40] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:07.183829 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7aab54-5c2a-4c92-adbf-146052151152 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:07.184128 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7aab54-5c2a-4c92-adbf-146052151152 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:07.184397 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7aab54-5c2a-4c92-adbf-146052151152 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:07.185958 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 19/41] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 19:52:07.192921 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc836359-2b00-498b-83a5-1138472d88b0 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:07.193209 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc836359-2b00-498b-83a5-1138472d88b0 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:07.193494 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc836359-2b00-498b-83a5-1138472d88b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:07.214963 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-cc836359-2b00-498b-83a5-1138472d88b0 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:07.216182 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-cc836359-2b00-498b-83a5-1138472d88b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:07.239009 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb01cd2-399d-45ef-9f8d-cddcac94e844 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:07.279046 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:07.282523 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8223025-acce-4325-8c18-5203264a7999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:07.283152 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8223025-acce-4325-8c18-5203264a7999 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:07.283435 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8223025-acce-4325-8c18-5203264a7999 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:07.283715 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8223025-acce-4325-8c18-5203264a7999 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:07.284474 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8223025-acce-4325-8c18-5203264a7999 None admin] RBAC: Adding query filter params (role.id=742dfaf63ef74207a2c9213411513925, scope.domain.id=default, user.id=55e5de29fcfb4f0abcb119685810b1da) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:07.285178 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8223025-acce-4325-8c18-5203264a7999 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:07.286286 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8223025-acce-4325-8c18-5203264a7999 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:07.295815 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 23/42] 10.222.0.13 () {56 vars in 1322 bytes} [Sun Oct 23 19:52:07 2022] GET /identity/v3/role_assignments?user.id=55e5de29fcfb4f0abcb119685810b1da&scope.domain.id=default&role.id=742dfaf63ef74207a2c9213411513925 => generated 530 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:07.323171 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 20/43] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 19:52:07.425431 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8223025-acce-4325-8c18-5203264a7999 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:07.460360 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:07.535645 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8223025-acce-4325-8c18-5203264a7999 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:07.569305 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b359d1-b47d-40c2-869d-05d4f3e78726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:07.569906 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b359d1-b47d-40c2-869d-05d4f3e78726 None admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:07.570187 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b359d1-b47d-40c2-869d-05d4f3e78726 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:07.570503 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b359d1-b47d-40c2-869d-05d4f3e78726 None admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:07.571173 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b359d1-b47d-40c2-869d-05d4f3e78726 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:07.572545 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b359d1-b47d-40c2-869d-05d4f3e78726 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:07.586050 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 24/44] 10.222.0.13 () {60 vars in 1131 bytes} [Sun Oct 23 19:52:07 2022] POST /identity/v3/roles => generated 223 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:52:07.591518 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc836359-2b00-498b-83a5-1138472d88b0 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:07.696375 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc836359-2b00-498b-83a5-1138472d88b0 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:07.735437 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b981b5e-ff65-4649-a2e6-9cf83b1b1c63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:07.736089 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b981b5e-ff65-4649-a2e6-9cf83b1b1c63 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:07.736374 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b981b5e-ff65-4649-a2e6-9cf83b1b1c63 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:07.736648 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b981b5e-ff65-4649-a2e6-9cf83b1b1c63 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:07.737792 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b981b5e-ff65-4649-a2e6-9cf83b1b1c63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:07.739438 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b981b5e-ff65-4649-a2e6-9cf83b1b1c63 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:07.741060 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 21/45] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:52:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:07.746754 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b359d1-b47d-40c2-869d-05d4f3e78726 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:07.841280 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:07.850801 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b359d1-b47d-40c2-869d-05d4f3e78726 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:07.884034 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177486ad-0db5-49f1-9baa-6244cb9344f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:07.884642 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177486ad-0db5-49f1-9baa-6244cb9344f3 None admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:07.884924 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177486ad-0db5-49f1-9baa-6244cb9344f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:07.885198 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177486ad-0db5-49f1-9baa-6244cb9344f3 None admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:07.885842 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177486ad-0db5-49f1-9baa-6244cb9344f3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:07.887303 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177486ad-0db5-49f1-9baa-6244cb9344f3 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:07.914686 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 25/46] 10.222.0.13 () {60 vars in 1141 bytes} [Sun Oct 23 19:52:07 2022] POST /identity/v3/projects => generated 312 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:52:10.026971 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9b10a-88b5-4fe2-930f-7478571440aa None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:10.027397 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9b10a-88b5-4fe2-930f-7478571440aa None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:10.027534 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9b10a-88b5-4fe2-930f-7478571440aa None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:10.028896 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 22/47] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:10.034767 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428230a-14a6-4371-ad01-688d8c682165 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:10.035159 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428230a-14a6-4371-ad01-688d8c682165 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:10.035325 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c428230a-14a6-4371-ad01-688d8c682165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:10.053850 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-c428230a-14a6-4371-ad01-688d8c682165 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:10.055088 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-c428230a-14a6-4371-ad01-688d8c682165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:10.118832 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 26/48] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:10.386505 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9b10a-88b5-4fe2-930f-7478571440aa None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:10.430428 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:10.485367 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a9b10a-88b5-4fe2-930f-7478571440aa None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:10.517860 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7df01d-5b07-4bfc-beb9-bb2150583557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:10.518508 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7df01d-5b07-4bfc-beb9-bb2150583557 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:10.518791 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7df01d-5b07-4bfc-beb9-bb2150583557 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:10.519063 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7df01d-5b07-4bfc-beb9-bb2150583557 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:10.520164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7df01d-5b07-4bfc-beb9-bb2150583557 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:10.521594 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7df01d-5b07-4bfc-beb9-bb2150583557 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:10.523165 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 23/49] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:52:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:10.528738 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428230a-14a6-4371-ad01-688d8c682165 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:10.599395 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:10.625043 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c428230a-14a6-4371-ad01-688d8c682165 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:10.657462 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b73ce7-6aa5-4ca7-a646-1b87a81c2265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:10.658011 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b73ce7-6aa5-4ca7-a646-1b87a81c2265 None admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:10.658314 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b73ce7-6aa5-4ca7-a646-1b87a81c2265 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:10.658592 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b73ce7-6aa5-4ca7-a646-1b87a81c2265 None admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:10.659234 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b73ce7-6aa5-4ca7-a646-1b87a81c2265 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:10.660680 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b73ce7-6aa5-4ca7-a646-1b87a81c2265 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:10.685318 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 27/50] 10.222.0.13 () {60 vars in 1140 bytes} [Sun Oct 23 19:52:10 2022] POST /identity/v3/projects => generated 298 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:52:12.786229 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac860214-6fa5-427c-bbd1-cd3947943eff None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:12.786792 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac860214-6fa5-427c-bbd1-cd3947943eff None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:12.786792 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac860214-6fa5-427c-bbd1-cd3947943eff None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:12.788160 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 24/51] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:12.793260 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8c6c3-c451-4f15-9ab9-ba9d76428dd3 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:12.793649 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8c6c3-c451-4f15-9ab9-ba9d76428dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:12.793816 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8c6c3-c451-4f15-9ab9-ba9d76428dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:12.812043 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-34b8c6c3-c451-4f15-9ab9-ba9d76428dd3 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:12.814056 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-34b8c6c3-c451-4f15-9ab9-ba9d76428dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:12.877887 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 28/52] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:13.147713 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac860214-6fa5-427c-bbd1-cd3947943eff None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:13.245422 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac860214-6fa5-427c-bbd1-cd3947943eff None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:13.256094 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:13.280293 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d93875-b99c-4989-ac6b-bc6f9659fb3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:13.280912 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d93875-b99c-4989-ac6b-bc6f9659fb3f None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:13.281194 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d93875-b99c-4989-ac6b-bc6f9659fb3f None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:13.281496 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d93875-b99c-4989-ac6b-bc6f9659fb3f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:13.282611 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d93875-b99c-4989-ac6b-bc6f9659fb3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:13.284046 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d93875-b99c-4989-ac6b-bc6f9659fb3f None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:13.285585 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 25/53] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:52:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:13.290981 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8c6c3-c451-4f15-9ab9-ba9d76428dd3 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:13.344981 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:13.388225 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8c6c3-c451-4f15-9ab9-ba9d76428dd3 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:13.420768 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc707ac-f0ed-4a8f-b512-d04eedf7d371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:13.421321 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc707ac-f0ed-4a8f-b512-d04eedf7d371 None admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:13.421606 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc707ac-f0ed-4a8f-b512-d04eedf7d371 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:13.421885 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc707ac-f0ed-4a8f-b512-d04eedf7d371 None admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:13.422575 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc707ac-f0ed-4a8f-b512-d04eedf7d371 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:13.424007 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc707ac-f0ed-4a8f-b512-d04eedf7d371 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:13.450784 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 29/54] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Oct 23 19:52:13 2022] POST /identity/v3/users => generated 275 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:52:16.658168 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66827533-e4f8-4c4f-812c-ab6d44d76d89 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:16.659072 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66827533-e4f8-4c4f-812c-ab6d44d76d89 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:16.659515 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66827533-e4f8-4c4f-812c-ab6d44d76d89 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:16.661020 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 26/55] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:16.666530 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fdbdba-68d9-4149-ab75-fcacb25a32ad None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:16.667251 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fdbdba-68d9-4149-ab75-fcacb25a32ad None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:16.667670 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fdbdba-68d9-4149-ab75-fcacb25a32ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:16.692604 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-58fdbdba-68d9-4149-ab75-fcacb25a32ad None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:16.694018 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-58fdbdba-68d9-4149-ab75-fcacb25a32ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:16.717175 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc4fab3-cd4a-420a-89ab-767f90e5132c None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:16.717773 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc4fab3-cd4a-420a-89ab-767f90e5132c None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:16.718273 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc4fab3-cd4a-420a-89ab-767f90e5132c None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:16.720200 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 27/56] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:16.725109 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f43201-6b1f-4726-b145-413c959fb8f5 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:16.725619 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f43201-6b1f-4726-b145-413c959fb8f5 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:16.726109 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f43201-6b1f-4726-b145-413c959fb8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:16.759839 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-69f43201-6b1f-4726-b145-413c959fb8f5 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:16.761045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-69f43201-6b1f-4726-b145-413c959fb8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:16.800922 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 30/57] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:16.873838 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 28/58] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 19:52:17.028590 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9dd65-ed3a-4280-bd59-2b906b9013eb None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:17.028881 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9dd65-ed3a-4280-bd59-2b906b9013eb None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:17.029163 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9dd65-ed3a-4280-bd59-2b906b9013eb None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:17.030591 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 31/59] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:17.047410 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cdf3b0-3687-48db-95af-d8bf37ce9b7a None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:17.047762 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cdf3b0-3687-48db-95af-d8bf37ce9b7a None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:17.048000 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cdf3b0-3687-48db-95af-d8bf37ce9b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:17.079497 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-f6cdf3b0-3687-48db-95af-d8bf37ce9b7a None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:17.080708 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f6cdf3b0-3687-48db-95af-d8bf37ce9b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:17.121925 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd9dd65-ed3a-4280-bd59-2b906b9013eb None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:17.194100 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 29/60] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:17.195774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c920d-0bf7-4771-8b15-43cab0308c9f None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:17.196292 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c920d-0bf7-4771-8b15-43cab0308c9f None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:17.196761 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c920d-0bf7-4771-8b15-43cab0308c9f None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:17.198212 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 30/61] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:17.202887 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:17.205301 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb820d0-3c84-4dfd-8b2b-c458af54711e None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:17.205924 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb820d0-3c84-4dfd-8b2b-c458af54711e None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:17.206287 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb820d0-3c84-4dfd-8b2b-c458af54711e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:17.233372 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-2cb820d0-3c84-4dfd-8b2b-c458af54711e None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:17.234918 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2cb820d0-3c84-4dfd-8b2b-c458af54711e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:17.266421 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:17.277686 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd9dd65-ed3a-4280-bd59-2b906b9013eb None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:17.316310 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 31/62] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:17.322267 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb820d0-3c84-4dfd-8b2b-c458af54711e None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:17.323336 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e2fbc-4e09-4ff1-93ce-d9d5c7fa23a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:17.323982 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e2fbc-4e09-4ff1-93ce-d9d5c7fa23a5 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:17.324264 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e2fbc-4e09-4ff1-93ce-d9d5c7fa23a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:17.324542 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0e2fbc-4e09-4ff1-93ce-d9d5c7fa23a5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:17.338329 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0e2fbc-4e09-4ff1-93ce-d9d5c7fa23a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:17.345523 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-2c0e2fbc-4e09-4ff1-93ce-d9d5c7fa23a5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 19:52:17.345523 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:17.345523 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:17.345523 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:17.345523 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:17.345523 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:17.345523 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:17.345523 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:17.345523 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:17.345523 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:17.345523 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:17.345523 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:17.345523 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:17.345523 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:17.345523 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:17.345523 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:17.346391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:17.346391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:17.346391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:17.346391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:17.346391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:17.346391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:17.346391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:17.346391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:17.346391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:17.346391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:17.346391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:17.346391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:17.346391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:17.346391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:17.346391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:17.346391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:17.347131 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:17.347131 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:17.347131 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:17.347131 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:17.347131 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:17.347131 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:17.347131 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:17.347131 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:17.347131 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:17.347131 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:17.347131 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 19:52:17.347131 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:52:17.349353 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 32/63] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:52:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:52:17.358269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e2fbc-4e09-4ff1-93ce-d9d5c7fa23a5 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:17.536799 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0e2fbc-4e09-4ff1-93ce-d9d5c7fa23a5 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:17.538774 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:17.574760 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b8c37-e6ab-4c56-9096-9b1605acd307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:17.575702 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b8c37-e6ab-4c56-9096-9b1605acd307 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:17.576197 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b8c37-e6ab-4c56-9096-9b1605acd307 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:17.576549 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1b8c37-e6ab-4c56-9096-9b1605acd307 None admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:17.577303 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1b8c37-e6ab-4c56-9096-9b1605acd307 None admin] RBAC: Adding query filter params (name=member) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:17.577611 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1b8c37-e6ab-4c56-9096-9b1605acd307 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:17.579038 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1b8c37-e6ab-4c56-9096-9b1605acd307 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:17.590478 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 33/64] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Oct 23 19:52:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:17.590478 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b8c37-e6ab-4c56-9096-9b1605acd307 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:17.597662 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb820d0-3c84-4dfd-8b2b-c458af54711e None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:17.632224 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e1fb7a-9504-48c7-9076-cde78d1f33d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:17.632856 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1fb7a-9504-48c7-9076-cde78d1f33d2 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:17.633139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1fb7a-9504-48c7-9076-cde78d1f33d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:17.633411 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e1fb7a-9504-48c7-9076-cde78d1f33d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:17.643354 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e1fb7a-9504-48c7-9076-cde78d1f33d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:17.648189 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-99e1fb7a-9504-48c7-9076-cde78d1f33d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:52:17.648189 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:17.648189 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:17.648189 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:17.648189 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:17.648189 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:17.648189 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:17.648189 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:17.648189 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:17.648189 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:17.648189 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:17.648189 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:17.648189 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:17.648189 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:17.648189 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:17.648189 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:17.649279 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:17.649279 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:17.649279 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:17.649279 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:17.649279 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:17.649279 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:17.649279 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:17.649279 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:17.649279 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:17.649279 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:17.649279 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:17.649279 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:17.649279 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:17.649279 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:17.649279 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:17.649279 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:17.650033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:17.650033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:17.650033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:17.650033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:17.650033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:17.650033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:17.650033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:17.650033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:17.650033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:17.650033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:17.650033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:52:17.650033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:52:17.650033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 32/65] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Oct 23 19:52:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:52:17.651073 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347eac5e-5afe-4904-befa-b9d790f445f6 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:17.651174 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347eac5e-5afe-4904-befa-b9d790f445f6 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:17.651433 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347eac5e-5afe-4904-befa-b9d790f445f6 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:17.652834 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 33/66] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:17.653366 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347eac5e-5afe-4904-befa-b9d790f445f6 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:17.682091 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:17.693629 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1b8c37-e6ab-4c56-9096-9b1605acd307 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:17.726568 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368fb80d-001d-4052-a052-55341ad0d3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:17.727212 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368fb80d-001d-4052-a052-55341ad0d3fd None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:17.727499 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368fb80d-001d-4052-a052-55341ad0d3fd None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:17.727778 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368fb80d-001d-4052-a052-55341ad0d3fd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:17.738763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368fb80d-001d-4052-a052-55341ad0d3fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:17.744168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-368fb80d-001d-4052-a052-55341ad0d3fd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 19:52:17.744168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:17.744168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:17.744168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:17.744168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:17.744168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:17.744168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:17.744168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:17.744168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:17.744168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:17.744168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:17.744168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:17.744168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:17.744168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:17.744168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:17.744168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:17.745170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:17.745170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:17.745170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:17.745170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:17.745170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:17.745170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:17.745170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:17.745170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:17.745170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:17.745170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:17.745170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:17.745170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:17.745170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:17.745170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:17.745170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:17.745170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:17.745949 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:17.745949 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:17.745949 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:17.745949 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:17.745949 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:17.745949 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:17.745949 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:17.745949 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:17.745949 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:17.745949 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:17.745949 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 19:52:17.745949 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:52:17.745949 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 34/67] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Oct 23 19:52:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:52:17.746593 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368fb80d-001d-4052-a052-55341ad0d3fd None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:17.759322 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347eac5e-5afe-4904-befa-b9d790f445f6 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:17.770996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:17.793650 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a0830-01da-4f55-909e-3b90553d218e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:17.794227 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a0830-01da-4f55-909e-3b90553d218e None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:17.794545 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a0830-01da-4f55-909e-3b90553d218e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:17.794827 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a0830-01da-4f55-909e-3b90553d218e None admin] PATH_INFO: `/v3/users/d158c81dd66f4f0181f82eb165a7b8a0` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:17.810187 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a0830-01da-4f55-909e-3b90553d218e None admin] RBAC: Authorizing `identity:get_user(user_id=d158c81dd66f4f0181f82eb165a7b8a0)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:17.811684 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a0830-01da-4f55-909e-3b90553d218e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:17.813904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 34/68] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:52:17 2022] GET /identity/v3/users/d158c81dd66f4f0181f82eb165a7b8a0 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:17.814720 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a0830-01da-4f55-909e-3b90553d218e None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:17.848024 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368fb80d-001d-4052-a052-55341ad0d3fd None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:17.881660 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc440e7-a254-485e-b8e1-eec4117b112d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:17.882318 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc440e7-a254-485e-b8e1-eec4117b112d None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:17.882605 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc440e7-a254-485e-b8e1-eec4117b112d None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:17.882882 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc440e7-a254-485e-b8e1-eec4117b112d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:17.893002 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc440e7-a254-485e-b8e1-eec4117b112d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:17.897486 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:17.898319 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-dfc440e7-a254-485e-b8e1-eec4117b112d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 19:52:17.898319 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:17.898319 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:17.898319 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:17.898319 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:17.898319 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:17.898319 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:17.898319 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:17.898319 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:17.898319 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:17.898319 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:17.898319 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:17.898319 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:17.898319 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:17.898319 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:17.898319 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:17.899149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:17.899149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:17.899149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:17.899149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:17.899149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:17.899149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:17.899149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:17.899149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:17.899149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:17.899149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:17.899149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:17.899149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:17.899149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:17.899149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:17.899149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:17.899149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:17.899888 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:17.899888 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:17.899888 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:17.899888 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:17.899888 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:17.899888 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:17.899888 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:17.899888 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:17.899888 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:17.899888 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:17.899888 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 19:52:17.899888 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:52:17.899888 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 35/69] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:52:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:52:17.901450 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf134f-c332-4916-a82a-961fc28f2361 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:17.901450 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf134f-c332-4916-a82a-961fc28f2361 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:17.901450 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcf134f-c332-4916-a82a-961fc28f2361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:17.913920 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a0830-01da-4f55-909e-3b90553d218e None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:17.919285 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-3bcf134f-c332-4916-a82a-961fc28f2361 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:17.920468 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-3bcf134f-c332-4916-a82a-961fc28f2361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:17.949421 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672cf69c-e0d4-4abe-9bce-cc566947d2aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:17.950034 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672cf69c-e0d4-4abe-9bce-cc566947d2aa None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:17.950348 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672cf69c-e0d4-4abe-9bce-cc566947d2aa None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:17.950629 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672cf69c-e0d4-4abe-9bce-cc566947d2aa None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:17.951299 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672cf69c-e0d4-4abe-9bce-cc566947d2aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:17.951608 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672cf69c-e0d4-4abe-9bce-cc566947d2aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:17.952996 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672cf69c-e0d4-4abe-9bce-cc566947d2aa None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:17.961458 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 35/70] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:52:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:17.963093 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672cf69c-e0d4-4abe-9bce-cc566947d2aa None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:17.986125 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 36/71] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:17.986125 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf134f-c332-4916-a82a-961fc28f2361 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:18.003368 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:18.076264 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672cf69c-e0d4-4abe-9bce-cc566947d2aa None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:18.098717 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcf134f-c332-4916-a82a-961fc28f2361 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:18.112780 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81997aaf-482a-4c13-9915-ace5a872168f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:18.113396 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81997aaf-482a-4c13-9915-ace5a872168f None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:18.113678 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81997aaf-482a-4c13-9915-ace5a872168f None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:18.113956 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81997aaf-482a-4c13-9915-ace5a872168f None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:18.114649 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81997aaf-482a-4c13-9915-ace5a872168f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:18.114955 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81997aaf-482a-4c13-9915-ace5a872168f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:18.116328 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81997aaf-482a-4c13-9915-ace5a872168f None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:18.128705 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 36/72] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:52:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:18.128705 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81997aaf-482a-4c13-9915-ace5a872168f None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:18.137479 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e9501-0fbf-4097-9e16-2d02065cf30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:18.138114 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e9501-0fbf-4097-9e16-2d02065cf30f None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:18.138428 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e9501-0fbf-4097-9e16-2d02065cf30f None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:18.138704 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e9501-0fbf-4097-9e16-2d02065cf30f None admin] PATH_INFO: `/v3/projects/0a195ad6e19d43ba991afe5458887d8c` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:18.139836 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6e9501-0fbf-4097-9e16-2d02065cf30f None admin] RBAC: Authorizing `identity:get_project(project_id=0a195ad6e19d43ba991afe5458887d8c)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:18.141436 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:18.141832 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6e9501-0fbf-4097-9e16-2d02065cf30f None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:18.143936 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 37/73] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Oct 23 19:52:17 2022] GET /identity/v3/projects/0a195ad6e19d43ba991afe5458887d8c => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:18.144777 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e9501-0fbf-4097-9e16-2d02065cf30f None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:18.226362 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:18.239049 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81997aaf-482a-4c13-9915-ace5a872168f None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:18.246480 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e9501-0fbf-4097-9e16-2d02065cf30f None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:18.272975 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fe4f7-4143-44f9-9671-98c7246af231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:18.273588 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fe4f7-4143-44f9-9671-98c7246af231 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:18.273876 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fe4f7-4143-44f9-9671-98c7246af231 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:18.274156 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364fe4f7-4143-44f9-9671-98c7246af231 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:18.274854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364fe4f7-4143-44f9-9671-98c7246af231 None admin] RBAC: Adding query filter params (name=member) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:18.275162 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364fe4f7-4143-44f9-9671-98c7246af231 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:18.276553 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364fe4f7-4143-44f9-9671-98c7246af231 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:18.283276 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9c7fc-4877-42eb-b2e2-cb5dd4e9bfd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:18.283909 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9c7fc-4877-42eb-b2e2-cb5dd4e9bfd6 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:18.284193 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9c7fc-4877-42eb-b2e2-cb5dd4e9bfd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:18.284473 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b9c7fc-4877-42eb-b2e2-cb5dd4e9bfd6 None admin] PATH_INFO: `/v3/users/55e5de29fcfb4f0abcb119685810b1da` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:18.285344 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 37/74] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Oct 23 19:52:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:18.285608 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b9c7fc-4877-42eb-b2e2-cb5dd4e9bfd6 None admin] RBAC: Authorizing `identity:get_user(user_id=55e5de29fcfb4f0abcb119685810b1da)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:18.286127 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fe4f7-4143-44f9-9671-98c7246af231 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:18.287066 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b9c7fc-4877-42eb-b2e2-cb5dd4e9bfd6 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:18.288660 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 38/75] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:52:18 2022] GET /identity/v3/users/55e5de29fcfb4f0abcb119685810b1da => generated 247 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:18.290403 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9c7fc-4877-42eb-b2e2-cb5dd4e9bfd6 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:18.354873 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:18.385865 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364fe4f7-4143-44f9-9671-98c7246af231 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:18.389472 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b9c7fc-4877-42eb-b2e2-cb5dd4e9bfd6 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:18.419573 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe13d78-2141-4cd5-bb60-655600e9c233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:18.420207 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe13d78-2141-4cd5-bb60-655600e9c233 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:18.420490 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe13d78-2141-4cd5-bb60-655600e9c233 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:18.420770 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe13d78-2141-4cd5-bb60-655600e9c233 None admin] PATH_INFO: `/v3/users/d158c81dd66f4f0181f82eb165a7b8a0` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:18.421897 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe13d78-2141-4cd5-bb60-655600e9c233 None admin] RBAC: Authorizing `identity:get_user(user_id=d158c81dd66f4f0181f82eb165a7b8a0)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:18.423027 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeee543-c01d-4589-82ab-6d1898fb63e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:18.423364 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe13d78-2141-4cd5-bb60-655600e9c233 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:18.423680 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeee543-c01d-4589-82ab-6d1898fb63e8 None admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:18.423961 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeee543-c01d-4589-82ab-6d1898fb63e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:18.424236 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeee543-c01d-4589-82ab-6d1898fb63e8 None admin] PATH_INFO: `/v3/projects/0a195ad6e19d43ba991afe5458887d8c/users/d158c81dd66f4f0181f82eb165a7b8a0/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:18.425167 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 38/76] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:52:18 2022] GET /identity/v3/users/d158c81dd66f4f0181f82eb165a7b8a0 => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:18.425709 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe13d78-2141-4cd5-bb60-655600e9c233 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:18.427519 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeee543-c01d-4589-82ab-6d1898fb63e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=0a195ad6e19d43ba991afe5458887d8c, user_id=d158c81dd66f4f0181f82eb165a7b8a0, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:18.429193 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeee543-c01d-4589-82ab-6d1898fb63e8 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:18.442334 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 39/77] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Oct 23 19:52:18 2022] PUT /identity/v3/projects/0a195ad6e19d43ba991afe5458887d8c/users/d158c81dd66f4f0181f82eb165a7b8a0/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 19:52:18.443135 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeee543-c01d-4589-82ab-6d1898fb63e8 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:18.457984 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:18.545096 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe13d78-2141-4cd5-bb60-655600e9c233 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:18.563287 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeee543-c01d-4589-82ab-6d1898fb63e8 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:18.584264 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86057007-5e05-4fe6-82ad-3dd36f53b5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:18.584925 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86057007-5e05-4fe6-82ad-3dd36f53b5a8 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:18.585304 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86057007-5e05-4fe6-82ad-3dd36f53b5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:18.585585 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86057007-5e05-4fe6-82ad-3dd36f53b5a8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:18.586775 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86057007-5e05-4fe6-82ad-3dd36f53b5a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:18.588256 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86057007-5e05-4fe6-82ad-3dd36f53b5a8 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:18.589937 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 39/78] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:52:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:18.590777 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86057007-5e05-4fe6-82ad-3dd36f53b5a8 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:18.606669 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:18.615283 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f32adb-6c28-4e17-ac0e-c3b7f384009b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:18.616495 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f32adb-6c28-4e17-ac0e-c3b7f384009b None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:18.616495 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f32adb-6c28-4e17-ac0e-c3b7f384009b None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:18.616495 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f32adb-6c28-4e17-ac0e-c3b7f384009b None admin] PATH_INFO: `/v3/users/d158c81dd66f4f0181f82eb165a7b8a0` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:18.617671 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f32adb-6c28-4e17-ac0e-c3b7f384009b None admin] RBAC: Authorizing `identity:get_user(user_id=d158c81dd66f4f0181f82eb165a7b8a0)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:18.619677 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f32adb-6c28-4e17-ac0e-c3b7f384009b None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:18.620722 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 40/79] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:52:18 2022] GET /identity/v3/users/d158c81dd66f4f0181f82eb165a7b8a0 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:18.621477 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f32adb-6c28-4e17-ac0e-c3b7f384009b None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:18.705985 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86057007-5e05-4fe6-82ad-3dd36f53b5a8 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:18.711086 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:18.736790 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f32adb-6c28-4e17-ac0e-c3b7f384009b None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:18.743700 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45fe17-fc6b-4205-99c4-1aa0a9bc18b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:18.744550 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45fe17-fc6b-4205-99c4-1aa0a9bc18b6 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:18.744762 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45fe17-fc6b-4205-99c4-1aa0a9bc18b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:18.745044 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e45fe17-fc6b-4205-99c4-1aa0a9bc18b6 None admin] PATH_INFO: `/v3/projects/0a195ad6e19d43ba991afe5458887d8c` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:18.746269 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45fe17-fc6b-4205-99c4-1aa0a9bc18b6 None admin] RBAC: Authorizing `identity:get_project(project_id=0a195ad6e19d43ba991afe5458887d8c)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:18.747758 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e45fe17-fc6b-4205-99c4-1aa0a9bc18b6 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:18.749633 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 40/80] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Oct 23 19:52:18 2022] GET /identity/v3/projects/0a195ad6e19d43ba991afe5458887d8c => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:18.751203 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45fe17-fc6b-4205-99c4-1aa0a9bc18b6 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:18.776473 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f704c-8b5b-4cf7-81a4-c51c70858c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:18.777114 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f704c-8b5b-4cf7-81a4-c51c70858c82 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:18.777397 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f704c-8b5b-4cf7-81a4-c51c70858c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:18.777919 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6f704c-8b5b-4cf7-81a4-c51c70858c82 None admin] PATH_INFO: `/v3/projects/0a195ad6e19d43ba991afe5458887d8c` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:18.778914 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f704c-8b5b-4cf7-81a4-c51c70858c82 None admin] RBAC: Authorizing `identity:get_project(project_id=0a195ad6e19d43ba991afe5458887d8c)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:18.780284 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6f704c-8b5b-4cf7-81a4-c51c70858c82 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:18.781982 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 41/81] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Oct 23 19:52:18 2022] GET /identity/v3/projects/0a195ad6e19d43ba991afe5458887d8c => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:18.782853 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f704c-8b5b-4cf7-81a4-c51c70858c82 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:18.839319 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:18.865778 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e45fe17-fc6b-4205-99c4-1aa0a9bc18b6 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:18.891913 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6f704c-8b5b-4cf7-81a4-c51c70858c82 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:18.900340 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf93b9-32b4-4ec3-81f8-39b959b040a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:18.901177 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf93b9-32b4-4ec3-81f8-39b959b040a6 None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:18.901629 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf93b9-32b4-4ec3-81f8-39b959b040a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:18.902072 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf93b9-32b4-4ec3-81f8-39b959b040a6 None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:18.902928 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdf93b9-32b4-4ec3-81f8-39b959b040a6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:18.904374 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdf93b9-32b4-4ec3-81f8-39b959b040a6 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:18.930434 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0251d4-b80a-475e-a11f-b3368661d6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:18.931071 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0251d4-b80a-475e-a11f-b3368661d6e0 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:18.931355 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0251d4-b80a-475e-a11f-b3368661d6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:18.931632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0251d4-b80a-475e-a11f-b3368661d6e0 None admin] PATH_INFO: `/v3/projects/a10f1afbb5964a199da6c71dddab1bfd` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:18.932766 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0251d4-b80a-475e-a11f-b3368661d6e0 None admin] RBAC: Authorizing `identity:get_project(project_id=a10f1afbb5964a199da6c71dddab1bfd)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:18.938620 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 41/82] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Oct 23 19:52:18 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:52:18.938620 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf93b9-32b4-4ec3-81f8-39b959b040a6 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:18.938860 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0251d4-b80a-475e-a11f-b3368661d6e0 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:18.938860 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 42/83] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Oct 23 19:52:18 2022] GET /identity/v3/projects/a10f1afbb5964a199da6c71dddab1bfd => generated 312 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:18.946499 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0251d4-b80a-475e-a11f-b3368661d6e0 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:18.965149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:19.084442 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0251d4-b80a-475e-a11f-b3368661d6e0 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:19.087618 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf93b9-32b4-4ec3-81f8-39b959b040a6 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:19.125561 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:19.127408 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93bb2da-fbf4-4080-b53b-f9bd206e6486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:19.128668 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93bb2da-fbf4-4080-b53b-f9bd206e6486 None admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:19.129014 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93bb2da-fbf4-4080-b53b-f9bd206e6486 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:19.129327 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93bb2da-fbf4-4080-b53b-f9bd206e6486 None admin] PATH_INFO: `/v3/projects/0a195ad6e19d43ba991afe5458887d8c/users/d158c81dd66f4f0181f82eb165a7b8a0/roles/2989d29f392d4d8c802354159891eb11` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:19.132650 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93bb2da-fbf4-4080-b53b-f9bd206e6486 None admin] RBAC: Authorizing `identity:create_grant(project_id=0a195ad6e19d43ba991afe5458887d8c, user_id=d158c81dd66f4f0181f82eb165a7b8a0, role_id=2989d29f392d4d8c802354159891eb11)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:19.134407 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93bb2da-fbf4-4080-b53b-f9bd206e6486 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:19.134548 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2e6a14-4c0b-418c-b537-2eec3c32c51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:19.135174 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e6a14-4c0b-418c-b537-2eec3c32c51b None admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:19.135618 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e6a14-4c0b-418c-b537-2eec3c32c51b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:19.136206 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2e6a14-4c0b-418c-b537-2eec3c32c51b None admin] PATH_INFO: `/v3/projects/0a195ad6e19d43ba991afe5458887d8c/users/55e5de29fcfb4f0abcb119685810b1da/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:19.139655 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2e6a14-4c0b-418c-b537-2eec3c32c51b None admin] RBAC: Authorizing `identity:create_grant(project_id=0a195ad6e19d43ba991afe5458887d8c, user_id=55e5de29fcfb4f0abcb119685810b1da, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:19.141860 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2e6a14-4c0b-418c-b537-2eec3c32c51b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:19.156010 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 43/84] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Oct 23 19:52:18 2022] PUT /identity/v3/projects/0a195ad6e19d43ba991afe5458887d8c/users/d158c81dd66f4f0181f82eb165a7b8a0/roles/2989d29f392d4d8c802354159891eb11 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 19:52:19.158086 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93bb2da-fbf4-4080-b53b-f9bd206e6486 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:19.164050 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 42/85] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Oct 23 19:52:18 2022] PUT /identity/v3/projects/0a195ad6e19d43ba991afe5458887d8c/users/55e5de29fcfb4f0abcb119685810b1da/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 19:52:19.300485 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:19.350042 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93bb2da-fbf4-4080-b53b-f9bd206e6486 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:19.415047 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe290f4-af70-4f76-9c47-3b3e6001e925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:19.415986 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe290f4-af70-4f76-9c47-3b3e6001e925 None admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:19.416440 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe290f4-af70-4f76-9c47-3b3e6001e925 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:19.416891 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe290f4-af70-4f76-9c47-3b3e6001e925 None admin] PATH_INFO: `/v3/projects/a10f1afbb5964a199da6c71dddab1bfd/users/d158c81dd66f4f0181f82eb165a7b8a0/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:19.420063 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe290f4-af70-4f76-9c47-3b3e6001e925 None admin] RBAC: Authorizing `identity:create_grant(project_id=a10f1afbb5964a199da6c71dddab1bfd, user_id=d158c81dd66f4f0181f82eb165a7b8a0, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:19.423264 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe290f4-af70-4f76-9c47-3b3e6001e925 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:19.446730 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 44/86] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Oct 23 19:52:19 2022] PUT /identity/v3/projects/a10f1afbb5964a199da6c71dddab1bfd/users/d158c81dd66f4f0181f82eb165a7b8a0/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 19:52:20.894045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa5ed2-9445-4f48-ad99-0b9a83280b6f None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:20.894580 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa5ed2-9445-4f48-ad99-0b9a83280b6f None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:20.894651 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fa5ed2-9445-4f48-ad99-0b9a83280b6f None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:20.896231 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 43/87] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:20.911031 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a23ac3-c00b-4357-9eda-2cc2c83cb3b6 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:20.911442 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a23ac3-c00b-4357-9eda-2cc2c83cb3b6 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:20.911615 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a23ac3-c00b-4357-9eda-2cc2c83cb3b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:20.934576 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-42a23ac3-c00b-4357-9eda-2cc2c83cb3b6 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:20.935837 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-42a23ac3-c00b-4357-9eda-2cc2c83cb3b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:21.015695 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 45/88] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 19:52:21.549312 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa5ed2-9445-4f48-ad99-0b9a83280b6f None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:21.688657 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fa5ed2-9445-4f48-ad99-0b9a83280b6f None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:21.738318 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e816e1c-7e6a-499e-ae4a-ae0b67c0f2fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:21.742107 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e816e1c-7e6a-499e-ae4a-ae0b67c0f2fa None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:21.742508 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e816e1c-7e6a-499e-ae4a-ae0b67c0f2fa None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:21.744411 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e816e1c-7e6a-499e-ae4a-ae0b67c0f2fa None admin] PATH_INFO: `/v3/roles/member` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:21.760278 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e816e1c-7e6a-499e-ae4a-ae0b67c0f2fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:21.767956 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-4e816e1c-7e6a-499e-ae4a-ae0b67c0f2fa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 19:52:21.767956 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:21.767956 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:21.767956 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:21.767956 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:21.767956 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:21.767956 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:21.767956 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:21.767956 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:21.767956 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:21.767956 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:21.767956 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:21.767956 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:21.767956 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:21.767956 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:21.767956 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:21.768797 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:21.768797 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:21.768797 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:21.768797 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:21.768797 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:21.768797 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:21.768797 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:21.768797 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:21.768797 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:21.768797 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:21.768797 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:21.768797 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:21.768797 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:21.768797 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:21.768797 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:21.768797 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:21.769552 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:21.769552 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:21.769552 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:21.769552 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:21.769552 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:21.769552 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:21.769552 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:21.769552 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:21.769552 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:21.769552 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:21.769552 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 19:52:21.769552 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:52:21.769552 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 44/89] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:52:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:52:21.775041 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a23ac3-c00b-4357-9eda-2cc2c83cb3b6 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:21.864261 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:21.911987 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a23ac3-c00b-4357-9eda-2cc2c83cb3b6 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:21.959228 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a78aa-d9ca-4fb9-b5cd-61a4486599d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:21.959897 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a78aa-d9ca-4fb9-b5cd-61a4486599d9 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:21.960185 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a78aa-d9ca-4fb9-b5cd-61a4486599d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:21.960462 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229a78aa-d9ca-4fb9-b5cd-61a4486599d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:21.961138 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229a78aa-d9ca-4fb9-b5cd-61a4486599d9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:21.961445 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229a78aa-d9ca-4fb9-b5cd-61a4486599d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:21.962861 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229a78aa-d9ca-4fb9-b5cd-61a4486599d9 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:21.978179 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 46/90] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Oct 23 19:52:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:21.983747 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e816e1c-7e6a-499e-ae4a-ae0b67c0f2fa None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:22.075142 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:22.137959 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e816e1c-7e6a-499e-ae4a-ae0b67c0f2fa None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:22.186028 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2884ce7e-f1e8-47da-b4c2-0959e30f1db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:22.186776 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884ce7e-f1e8-47da-b4c2-0959e30f1db4 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:22.187058 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884ce7e-f1e8-47da-b4c2-0959e30f1db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:22.187334 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2884ce7e-f1e8-47da-b4c2-0959e30f1db4 None admin] PATH_INFO: `/v3/users/d158c81dd66f4f0181f82eb165a7b8a0` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:22.189072 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2884ce7e-f1e8-47da-b4c2-0959e30f1db4 None admin] RBAC: Authorizing `identity:get_user(user_id=d158c81dd66f4f0181f82eb165a7b8a0)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:22.190577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2884ce7e-f1e8-47da-b4c2-0959e30f1db4 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:22.192234 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 45/91] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:52:21 2022] GET /identity/v3/users/d158c81dd66f4f0181f82eb165a7b8a0 => generated 275 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:22.198154 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a78aa-d9ca-4fb9-b5cd-61a4486599d9 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:22.349953 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229a78aa-d9ca-4fb9-b5cd-61a4486599d9 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:22.376786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:22.385679 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a0dd9f-01d3-4aff-afda-45fbaa48fef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:22.386328 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0dd9f-01d3-4aff-afda-45fbaa48fef6 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:22.386615 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0dd9f-01d3-4aff-afda-45fbaa48fef6 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:22.386890 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a0dd9f-01d3-4aff-afda-45fbaa48fef6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:22.411232 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ea9ef-123f-41f7-9091-a6ce74df9753 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:22.411505 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ea9ef-123f-41f7-9091-a6ce74df9753 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:22.411725 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ea9ef-123f-41f7-9091-a6ce74df9753 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:22.413077 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 46/92] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:22.419189 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc122f-ca15-44fd-95ba-fafc0e655af5 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:22.419481 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc122f-ca15-44fd-95ba-fafc0e655af5 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:22.419760 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cc122f-ca15-44fd-95ba-fafc0e655af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:22.451667 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-50cc122f-ca15-44fd-95ba-fafc0e655af5 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:22.452886 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-50cc122f-ca15-44fd-95ba-fafc0e655af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:22.477617 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a0dd9f-01d3-4aff-afda-45fbaa48fef6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:22.479894 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a0dd9f-01d3-4aff-afda-45fbaa48fef6 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:22.532907 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 47/93] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 19:52:22.534356 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19c131-e98d-4b4d-8b80-e83231fe51a6 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:22.534661 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19c131-e98d-4b4d-8b80-e83231fe51a6 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:22.534944 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd19c131-e98d-4b4d-8b80-e83231fe51a6 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:22.536278 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 48/94] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:22.542311 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2525035-9b7a-4a9b-8834-37367d49ceb9 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:22.542604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2525035-9b7a-4a9b-8834-37367d49ceb9 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:22.542885 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2525035-9b7a-4a9b-8834-37367d49ceb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:22.563437 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-d2525035-9b7a-4a9b-8834-37367d49ceb9 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:22.565932 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d2525035-9b7a-4a9b-8834-37367d49ceb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:22.655819 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:22.694138 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 49/96] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:22.694338 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 47/96] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Oct 23 19:52:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 19:52:22.698085 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2525035-9b7a-4a9b-8834-37367d49ceb9 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:22.725584 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd026835-9460-4830-8ffa-639e82151f56 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:22.725919 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd026835-9460-4830-8ffa-639e82151f56 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:22.726147 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd026835-9460-4830-8ffa-639e82151f56 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:22.727528 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 48/97] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 19:52:22.732922 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3162f6-7099-4bda-95fe-8a0ac932dd6a None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:22.733206 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3162f6-7099-4bda-95fe-8a0ac932dd6a None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:22.733479 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3162f6-7099-4bda-95fe-8a0ac932dd6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:22.767943 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-2b3162f6-7099-4bda-95fe-8a0ac932dd6a None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:22.769135 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-2b3162f6-7099-4bda-95fe-8a0ac932dd6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:22.820814 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:22.913179 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 49/98] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:22.914048 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3162f6-7099-4bda-95fe-8a0ac932dd6a None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:22.926264 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2525035-9b7a-4a9b-8834-37367d49ceb9 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:23.015684 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c4716-f22f-418e-afb2-094cda8f52f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:23.017102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c4716-f22f-418e-afb2-094cda8f52f3 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:23.017589 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c4716-f22f-418e-afb2-094cda8f52f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:23.018010 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c4716-f22f-418e-afb2-094cda8f52f3 None admin] PATH_INFO: `/v3/projects/0a195ad6e19d43ba991afe5458887d8c` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:23.021365 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80c4716-f22f-418e-afb2-094cda8f52f3 None admin] RBAC: Authorizing `identity:get_project(project_id=0a195ad6e19d43ba991afe5458887d8c)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:23.023094 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80c4716-f22f-418e-afb2-094cda8f52f3 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:23.025067 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 50/99] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Oct 23 19:52:22 2022] GET /identity/v3/projects/0a195ad6e19d43ba991afe5458887d8c => generated 298 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:23.026305 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3162f6-7099-4bda-95fe-8a0ac932dd6a None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:23.030545 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c228f-01a0-4885-98e6-ea3ca11c2b04 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:23.030874 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c228f-01a0-4885-98e6-ea3ca11c2b04 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:23.031163 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3c228f-01a0-4885-98e6-ea3ca11c2b04 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:23.033264 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 51/100] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:23.034006 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c228f-01a0-4885-98e6-ea3ca11c2b04 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:23.069211 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e4633-b541-43a0-80ec-e8013a333593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:23.070838 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e4633-b541-43a0-80ec-e8013a333593 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:23.071302 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e4633-b541-43a0-80ec-e8013a333593 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:23.072027 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e4633-b541-43a0-80ec-e8013a333593 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:23.084992 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e4633-b541-43a0-80ec-e8013a333593 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:23.092106 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-590e4633-b541-43a0-80ec-e8013a333593 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:52:23.092106 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:23.092106 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:23.092106 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:23.092106 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:23.092106 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:23.092106 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:23.092106 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:23.092106 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:23.092106 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:23.092106 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:23.092106 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:23.092106 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:23.092106 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:23.092106 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:23.092106 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:23.092931 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:23.092931 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:23.092931 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:23.092931 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:23.092931 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:23.092931 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:23.092931 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:23.092931 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:23.092931 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:23.092931 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:23.092931 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:23.092931 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:23.092931 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:23.092931 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:23.092931 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:23.092931 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:23.094311 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:23.094311 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:23.094311 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:23.094311 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:23.094311 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:23.094311 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:23.094311 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:23.094311 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:23.094311 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:23.094311 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:23.094311 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:52:23.094311 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:52:23.096206 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 50/101] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Oct 23 19:52:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:52:23.101346 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e4633-b541-43a0-80ec-e8013a333593 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:23.110707 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:23.117501 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:23.167733 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3c228f-01a0-4885-98e6-ea3ca11c2b04 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:23.212794 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8e478e-1e6f-4a7e-accc-b0e264e0aa7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:23.213434 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e478e-1e6f-4a7e-accc-b0e264e0aa7f None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:23.213763 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e478e-1e6f-4a7e-accc-b0e264e0aa7f None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:23.214121 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8e478e-1e6f-4a7e-accc-b0e264e0aa7f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:23.227626 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8e478e-1e6f-4a7e-accc-b0e264e0aa7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:23.228560 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e4633-b541-43a0-80ec-e8013a333593 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:23.233526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-0f8e478e-1e6f-4a7e-accc-b0e264e0aa7f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 19:52:23.233526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:23.233526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:23.233526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:23.233526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:23.233526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:23.233526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:23.233526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:23.233526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:23.233526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:23.233526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:23.233526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:23.233526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:23.233526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:23.233526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:23.233526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:23.235210 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:23.235210 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:23.235210 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:23.235210 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:23.235210 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:23.235210 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:23.235210 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:23.235210 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:23.235210 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:23.235210 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:23.235210 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:23.235210 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:23.235210 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:23.235210 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:23.235210 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:23.235210 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:23.236362 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:23.236362 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:23.236362 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:23.236362 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:23.236362 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:23.236362 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:23.236362 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:23.236362 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:23.236362 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:23.236362 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:23.236362 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 19:52:23.236362 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:52:23.236362 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 52/102] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:52:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:52:23.236362 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3540c9-169a-4046-bd18-cfa6bb1c1469 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:23.237064 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3540c9-169a-4046-bd18-cfa6bb1c1469 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:23.237064 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3540c9-169a-4046-bd18-cfa6bb1c1469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:23.255736 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-9d3540c9-169a-4046-bd18-cfa6bb1c1469 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:23.256934 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9d3540c9-169a-4046-bd18-cfa6bb1c1469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:23.269455 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0aea05-227d-4d43-b439-6e2ace3164d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:23.270081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0aea05-227d-4d43-b439-6e2ace3164d1 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:23.270314 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0aea05-227d-4d43-b439-6e2ace3164d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:23.270594 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0aea05-227d-4d43-b439-6e2ace3164d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:23.271332 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0aea05-227d-4d43-b439-6e2ace3164d1 None admin] RBAC: Adding query filter params (role.id=69610ceae19c4a2b963433e2a9f5bb26, scope.project.id=0a195ad6e19d43ba991afe5458887d8c, user.id=d158c81dd66f4f0181f82eb165a7b8a0) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:23.271641 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0aea05-227d-4d43-b439-6e2ace3164d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:23.273089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0aea05-227d-4d43-b439-6e2ace3164d1 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:23.279337 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 51/103] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Oct 23 19:52:23 2022] GET /identity/v3/role_assignments?user.id=d158c81dd66f4f0181f82eb165a7b8a0&scope.project.id=0a195ad6e19d43ba991afe5458887d8c&role.id=69610ceae19c4a2b963433e2a9f5bb26 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:23.280054 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0aea05-227d-4d43-b439-6e2ace3164d1 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:23.350286 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 53/104] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:23.350286 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3540c9-169a-4046-bd18-cfa6bb1c1469 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:23.406657 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:23.411318 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:23.417548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0aea05-227d-4d43-b439-6e2ace3164d1 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:23.472580 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8384aff7-d66a-4c23-90a2-ff5932a017d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:23.473593 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384aff7-d66a-4c23-90a2-ff5932a017d9 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:23.474040 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384aff7-d66a-4c23-90a2-ff5932a017d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:23.474353 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8384aff7-d66a-4c23-90a2-ff5932a017d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:23.475051 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8384aff7-d66a-4c23-90a2-ff5932a017d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:23.475418 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8384aff7-d66a-4c23-90a2-ff5932a017d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:23.476815 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8384aff7-d66a-4c23-90a2-ff5932a017d9 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:23.486924 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 52/105] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:52:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:23.487775 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8384aff7-d66a-4c23-90a2-ff5932a017d9 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:23.494560 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3540c9-169a-4046-bd18-cfa6bb1c1469 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:23.546334 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee7a0f2-bac6-46ca-a5e2-72029b61d0b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:23.547542 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7a0f2-bac6-46ca-a5e2-72029b61d0b9 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:23.547893 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7a0f2-bac6-46ca-a5e2-72029b61d0b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:23.548238 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee7a0f2-bac6-46ca-a5e2-72029b61d0b9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:23.558861 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee7a0f2-bac6-46ca-a5e2-72029b61d0b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:23.564062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-0ee7a0f2-bac6-46ca-a5e2-72029b61d0b9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 19:52:23.564062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:23.564062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:23.564062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:23.564062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:23.564062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:23.564062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:23.564062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:23.564062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:23.564062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:23.564062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:23.564062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:23.564062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:23.564062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:23.564062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:23.564062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:23.564909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:23.564909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:23.564909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:23.564909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:23.564909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:23.564909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:23.564909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:23.564909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:23.564909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:23.564909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:23.564909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:23.564909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:23.564909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:23.564909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:23.564909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:23.564909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:23.565688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:23.565688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:23.565688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:23.565688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:23.565688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:23.565688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:23.565688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:23.565688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:23.565688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:23.565688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:23.565688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 19:52:23.565688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:52:23.565688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 54/106] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:52:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:52:23.566367 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba7818-4ee4-4993-81df-f316a8663964 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:23.567379 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba7818-4ee4-4993-81df-f316a8663964 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:23.567379 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ba7818-4ee4-4993-81df-f316a8663964 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:23.570355 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 55/107] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:23.570355 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ba7818-4ee4-4993-81df-f316a8663964 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:23.609167 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8384aff7-d66a-4c23-90a2-ff5932a017d9 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:23.644932 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070e405d-22bc-4708-8016-a16f31cbf3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:23.645549 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e405d-22bc-4708-8016-a16f31cbf3ed None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:23.645838 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e405d-22bc-4708-8016-a16f31cbf3ed None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:23.646110 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070e405d-22bc-4708-8016-a16f31cbf3ed None admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:23.646810 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e405d-22bc-4708-8016-a16f31cbf3ed None admin] RBAC: Adding query filter params (name=reader) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:23.647120 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e405d-22bc-4708-8016-a16f31cbf3ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:23.648504 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070e405d-22bc-4708-8016-a16f31cbf3ed None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:23.657288 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 53/108] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Oct 23 19:52:23 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:23.658459 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e82570-09e6-4eff-9b74-d3ea2fad3677 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:23.658751 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e82570-09e6-4eff-9b74-d3ea2fad3677 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:23.659029 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e82570-09e6-4eff-9b74-d3ea2fad3677 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:23.665940 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:23.675858 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:23.677574 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-40e82570-09e6-4eff-9b74-d3ea2fad3677 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:23.678800 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-40e82570-09e6-4eff-9b74-d3ea2fad3677 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:23.680918 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ba7818-4ee4-4993-81df-f316a8663964 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:23.714208 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0a93c-66d2-4cde-b94f-462a839bf2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:23.714865 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0a93c-66d2-4cde-b94f-462a839bf2fd None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:23.715153 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0a93c-66d2-4cde-b94f-462a839bf2fd None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:23.715434 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0a93c-66d2-4cde-b94f-462a839bf2fd None admin] PATH_INFO: `/v3/users/55e5de29fcfb4f0abcb119685810b1da` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:23.716565 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd0a93c-66d2-4cde-b94f-462a839bf2fd None admin] RBAC: Authorizing `identity:get_user(user_id=55e5de29fcfb4f0abcb119685810b1da)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:23.718006 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd0a93c-66d2-4cde-b94f-462a839bf2fd None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:23.720152 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 56/109] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:52:23 2022] GET /identity/v3/users/55e5de29fcfb4f0abcb119685810b1da => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:23.720926 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0a93c-66d2-4cde-b94f-462a839bf2fd None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:23.744982 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 54/110] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:23.746082 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e82570-09e6-4eff-9b74-d3ea2fad3677 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:23.834414 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd0a93c-66d2-4cde-b94f-462a839bf2fd None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:23.848857 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e82570-09e6-4eff-9b74-d3ea2fad3677 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:23.870311 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97693b-039c-4443-98e3-9f8e0af5f3a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:23.870923 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97693b-039c-4443-98e3-9f8e0af5f3a5 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:23.871208 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97693b-039c-4443-98e3-9f8e0af5f3a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:23.871485 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97693b-039c-4443-98e3-9f8e0af5f3a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:23.872143 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f97693b-039c-4443-98e3-9f8e0af5f3a5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:23.872449 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f97693b-039c-4443-98e3-9f8e0af5f3a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:23.873826 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f97693b-039c-4443-98e3-9f8e0af5f3a5 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:23.882382 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 57/111] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Oct 23 19:52:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:23.887224 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97693b-039c-4443-98e3-9f8e0af5f3a5 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:23.888880 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c5de1c-01d4-4a9d-9c8d-e693892e8367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:23.889535 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5de1c-01d4-4a9d-9c8d-e693892e8367 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:23.889816 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5de1c-01d4-4a9d-9c8d-e693892e8367 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:23.890088 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c5de1c-01d4-4a9d-9c8d-e693892e8367 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:23.892186 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:23.902823 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c5de1c-01d4-4a9d-9c8d-e693892e8367 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:23.908340 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-f8c5de1c-01d4-4a9d-9c8d-e693892e8367 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 19:52:23.908340 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:23.908340 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:23.908340 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:23.908340 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:23.908340 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:23.908340 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:23.908340 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:23.908340 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:23.908340 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:23.908340 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:23.908340 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:23.908340 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:23.908340 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:23.908340 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:23.908340 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:23.909170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:23.909170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:23.909170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:23.909170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:23.909170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:23.909170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:23.909170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:23.909170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:23.909170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:23.909170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:23.909170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:23.909170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:23.909170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:23.909170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:23.909170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:23.909170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:23.910395 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:23.910395 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:23.910395 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:23.910395 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:23.910395 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:23.910395 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:23.910395 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:23.910395 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:23.910395 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:23.910395 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:23.910395 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 19:52:23.910395 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:52:23.910395 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 55/112] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Oct 23 19:52:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:52:23.910395 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c5de1c-01d4-4a9d-9c8d-e693892e8367 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:23.920425 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:23.997221 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97693b-039c-4443-98e3-9f8e0af5f3a5 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:24.014563 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c5de1c-01d4-4a9d-9c8d-e693892e8367 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:24.032020 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51170baf-7111-4187-87f0-8ac182e3984e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:24.032662 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51170baf-7111-4187-87f0-8ac182e3984e None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:24.032945 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51170baf-7111-4187-87f0-8ac182e3984e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:24.033223 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51170baf-7111-4187-87f0-8ac182e3984e None admin] PATH_INFO: `/v3/users/c569ecb5d4b2433abf6c464f88cf2ec1` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:24.046555 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51170baf-7111-4187-87f0-8ac182e3984e None admin] RBAC: Authorizing `identity:get_user(user_id=c569ecb5d4b2433abf6c464f88cf2ec1)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:24.048551 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51170baf-7111-4187-87f0-8ac182e3984e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:24.050180 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 58/113] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:52:23 2022] GET /identity/v3/users/c569ecb5d4b2433abf6c464f88cf2ec1 => generated 289 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:24.054968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51170baf-7111-4187-87f0-8ac182e3984e None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:24.055251 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71eae7-f06e-47cd-ad37-ec93b6211bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:24.055887 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71eae7-f06e-47cd-ad37-ec93b6211bf4 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:24.056155 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71eae7-f06e-47cd-ad37-ec93b6211bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:24.056449 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71eae7-f06e-47cd-ad37-ec93b6211bf4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:24.115292 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:24.122554 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f71eae7-f06e-47cd-ad37-ec93b6211bf4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:24.124039 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f71eae7-f06e-47cd-ad37-ec93b6211bf4 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:24.138400 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:24.154343 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51170baf-7111-4187-87f0-8ac182e3984e None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:24.189274 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6209e772-2be5-4053-9fb8-5aedaf6e2428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:24.189820 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209e772-2be5-4053-9fb8-5aedaf6e2428 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:24.190111 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209e772-2be5-4053-9fb8-5aedaf6e2428 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:24.190423 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209e772-2be5-4053-9fb8-5aedaf6e2428 None admin] PATH_INFO: `/v3/users/d158c81dd66f4f0181f82eb165a7b8a0` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:24.191597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6209e772-2be5-4053-9fb8-5aedaf6e2428 None admin] RBAC: Authorizing `identity:get_user(user_id=d158c81dd66f4f0181f82eb165a7b8a0)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:24.193043 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6209e772-2be5-4053-9fb8-5aedaf6e2428 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:24.194675 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 59/114] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:52:24 2022] GET /identity/v3/users/d158c81dd66f4f0181f82eb165a7b8a0 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:24.195445 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6209e772-2be5-4053-9fb8-5aedaf6e2428 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:24.224604 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 56/115] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Oct 23 19:52:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 19:52:24.225505 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71eae7-f06e-47cd-ad37-ec93b6211bf4 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:24.297297 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6209e772-2be5-4053-9fb8-5aedaf6e2428 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:24.315066 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:24.321824 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71eae7-f06e-47cd-ad37-ec93b6211bf4 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:24.331126 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0997512b-e167-4f55-a96e-b3b1d19d48fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:24.331732 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997512b-e167-4f55-a96e-b3b1d19d48fc None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:24.332026 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997512b-e167-4f55-a96e-b3b1d19d48fc None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:24.332299 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0997512b-e167-4f55-a96e-b3b1d19d48fc None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:24.332992 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0997512b-e167-4f55-a96e-b3b1d19d48fc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:24.333566 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0997512b-e167-4f55-a96e-b3b1d19d48fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:24.334693 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0997512b-e167-4f55-a96e-b3b1d19d48fc None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:24.342569 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 60/116] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:52:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:24.343430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0997512b-e167-4f55-a96e-b3b1d19d48fc None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:24.361276 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:24.361576 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af27420-b3bd-41fe-80b3-d4198e82ac16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:24.362392 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af27420-b3bd-41fe-80b3-d4198e82ac16 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:24.362614 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af27420-b3bd-41fe-80b3-d4198e82ac16 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:24.363068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af27420-b3bd-41fe-80b3-d4198e82ac16 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:24.364237 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af27420-b3bd-41fe-80b3-d4198e82ac16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:24.365731 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af27420-b3bd-41fe-80b3-d4198e82ac16 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:24.370432 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 57/117] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:52:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:24.370432 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af27420-b3bd-41fe-80b3-d4198e82ac16 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:24.447122 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0997512b-e167-4f55-a96e-b3b1d19d48fc None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:24.465742 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af27420-b3bd-41fe-80b3-d4198e82ac16 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:24.480557 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb04f4-b874-461c-a239-107ab386709e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:24.481195 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb04f4-b874-461c-a239-107ab386709e None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:24.481483 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb04f4-b874-461c-a239-107ab386709e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:24.481765 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeb04f4-b874-461c-a239-107ab386709e None admin] PATH_INFO: `/v3/projects/0a195ad6e19d43ba991afe5458887d8c` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:24.482910 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb04f4-b874-461c-a239-107ab386709e None admin] RBAC: Authorizing `identity:get_project(project_id=0a195ad6e19d43ba991afe5458887d8c)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:24.484396 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aeb04f4-b874-461c-a239-107ab386709e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:24.486704 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 61/118] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Oct 23 19:52:24 2022] GET /identity/v3/projects/0a195ad6e19d43ba991afe5458887d8c => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:24.488001 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb04f4-b874-461c-a239-107ab386709e None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:24.506221 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d75dcd-eb8b-4e61-9c8b-07cce9bc21c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:24.506859 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d75dcd-eb8b-4e61-9c8b-07cce9bc21c4 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:24.507139 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d75dcd-eb8b-4e61-9c8b-07cce9bc21c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:24.507413 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d75dcd-eb8b-4e61-9c8b-07cce9bc21c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:24.541469 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:24.576848 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:24.579995 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d75dcd-eb8b-4e61-9c8b-07cce9bc21c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:24.581476 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d75dcd-eb8b-4e61-9c8b-07cce9bc21c4 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:24.593158 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeb04f4-b874-461c-a239-107ab386709e None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:24.626713 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa59f0-56ed-410f-a8c3-2db227026f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:24.627275 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa59f0-56ed-410f-a8c3-2db227026f3f None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:24.627558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa59f0-56ed-410f-a8c3-2db227026f3f None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:24.627835 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa59f0-56ed-410f-a8c3-2db227026f3f None admin] PATH_INFO: `/v3/projects/0a195ad6e19d43ba991afe5458887d8c` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:24.628942 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa59f0-56ed-410f-a8c3-2db227026f3f None admin] RBAC: Authorizing `identity:get_project(project_id=0a195ad6e19d43ba991afe5458887d8c)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:24.630407 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fa59f0-56ed-410f-a8c3-2db227026f3f None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:24.632049 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 62/119] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Oct 23 19:52:24 2022] GET /identity/v3/projects/0a195ad6e19d43ba991afe5458887d8c => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:24.632807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa59f0-56ed-410f-a8c3-2db227026f3f None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:24.678187 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 58/120] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Oct 23 19:52:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 19:52:24.678187 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d75dcd-eb8b-4e61-9c8b-07cce9bc21c4 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:24.731319 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fa59f0-56ed-410f-a8c3-2db227026f3f None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:24.737893 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:24.765027 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cadfa1f-03a6-414f-b49a-11c227c02319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:24.766195 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cadfa1f-03a6-414f-b49a-11c227c02319 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:24.766504 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cadfa1f-03a6-414f-b49a-11c227c02319 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:24.766780 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cadfa1f-03a6-414f-b49a-11c227c02319 None admin] PATH_INFO: `/v3/users/d158c81dd66f4f0181f82eb165a7b8a0` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:24.767899 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cadfa1f-03a6-414f-b49a-11c227c02319 None admin] RBAC: Authorizing `identity:get_user(user_id=d158c81dd66f4f0181f82eb165a7b8a0)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:24.769338 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cadfa1f-03a6-414f-b49a-11c227c02319 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:24.770961 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 63/121] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:52:24 2022] GET /identity/v3/users/d158c81dd66f4f0181f82eb165a7b8a0 => generated 275 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:24.776162 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cadfa1f-03a6-414f-b49a-11c227c02319 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:24.776297 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d75dcd-eb8b-4e61-9c8b-07cce9bc21c4 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:24.797702 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:24.809339 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c3164-b297-4fca-9a78-4ca354e519ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:24.809952 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c3164-b297-4fca-9a78-4ca354e519ba None admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:24.810235 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c3164-b297-4fca-9a78-4ca354e519ba None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:24.810546 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9c3164-b297-4fca-9a78-4ca354e519ba None admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:24.811208 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c3164-b297-4fca-9a78-4ca354e519ba None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:24.812936 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9c3164-b297-4fca-9a78-4ca354e519ba None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:24.839001 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 59/122] 10.222.0.13 () {60 vars in 1141 bytes} [Sun Oct 23 19:52:24 2022] POST /identity/v3/projects => generated 302 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:52:24.839810 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c3164-b297-4fca-9a78-4ca354e519ba None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:24.885356 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cadfa1f-03a6-414f-b49a-11c227c02319 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:24.923962 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38257e64-d1a2-4722-9336-14dc3c65ee35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:24.924666 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38257e64-d1a2-4722-9336-14dc3c65ee35 None admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:24.924961 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38257e64-d1a2-4722-9336-14dc3c65ee35 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:24.925248 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38257e64-d1a2-4722-9336-14dc3c65ee35 None admin] PATH_INFO: `/v3/projects/0a195ad6e19d43ba991afe5458887d8c/users/c569ecb5d4b2433abf6c464f88cf2ec1/roles/54768ecdabbd443ca182db572b13b3ca` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:24.927309 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38257e64-d1a2-4722-9336-14dc3c65ee35 None admin] RBAC: Authorizing `identity:create_grant(project_id=0a195ad6e19d43ba991afe5458887d8c, user_id=c569ecb5d4b2433abf6c464f88cf2ec1, role_id=54768ecdabbd443ca182db572b13b3ca)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:24.928783 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38257e64-d1a2-4722-9336-14dc3c65ee35 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:24.945207 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 64/123] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Oct 23 19:52:24 2022] PUT /identity/v3/projects/0a195ad6e19d43ba991afe5458887d8c/users/c569ecb5d4b2433abf6c464f88cf2ec1/roles/54768ecdabbd443ca182db572b13b3ca => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 19:52:24.946747 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38257e64-d1a2-4722-9336-14dc3c65ee35 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:24.970528 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9c3164-b297-4fca-9a78-4ca354e519ba None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:24.987765 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:25.012849 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6864ca-498f-4d48-b866-e61420b16167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:25.013537 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6864ca-498f-4d48-b866-e61420b16167 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:25.013830 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6864ca-498f-4d48-b866-e61420b16167 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:25.014105 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6864ca-498f-4d48-b866-e61420b16167 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:25.014971 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6864ca-498f-4d48-b866-e61420b16167 None admin] RBAC: Adding query filter params (role.id=742dfaf63ef74207a2c9213411513925, scope.project.id=0a195ad6e19d43ba991afe5458887d8c, user.id=55e5de29fcfb4f0abcb119685810b1da) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:25.015503 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6864ca-498f-4d48-b866-e61420b16167 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:25.017334 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6864ca-498f-4d48-b866-e61420b16167 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:25.025370 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 60/124] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Oct 23 19:52:24 2022] GET /identity/v3/role_assignments?user.id=55e5de29fcfb4f0abcb119685810b1da&scope.project.id=0a195ad6e19d43ba991afe5458887d8c&role.id=742dfaf63ef74207a2c9213411513925 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:25.027182 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6864ca-498f-4d48-b866-e61420b16167 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:25.093427 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:25.127718 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38257e64-d1a2-4722-9336-14dc3c65ee35 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:25.163506 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6864ca-498f-4d48-b866-e61420b16167 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:25.179003 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef813d5d-2800-46a7-947e-a7dc56655c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:25.179685 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef813d5d-2800-46a7-947e-a7dc56655c8e None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:25.181621 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef813d5d-2800-46a7-947e-a7dc56655c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:25.181621 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef813d5d-2800-46a7-947e-a7dc56655c8e None admin] PATH_INFO: `/v3/projects/a10f1afbb5964a199da6c71dddab1bfd` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:25.181807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef813d5d-2800-46a7-947e-a7dc56655c8e None admin] RBAC: Authorizing `identity:get_project(project_id=a10f1afbb5964a199da6c71dddab1bfd)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:25.186280 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef813d5d-2800-46a7-947e-a7dc56655c8e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:25.186456 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 65/125] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Oct 23 19:52:24 2022] GET /identity/v3/projects/a10f1afbb5964a199da6c71dddab1bfd => generated 312 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:25.194494 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef813d5d-2800-46a7-947e-a7dc56655c8e None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:25.209073 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb0ac6-cc9d-4fcb-a459-eb6810786b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:25.209897 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb0ac6-cc9d-4fcb-a459-eb6810786b14 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:25.210269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb0ac6-cc9d-4fcb-a459-eb6810786b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:25.210678 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb0ac6-cc9d-4fcb-a459-eb6810786b14 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:25.277535 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:25.308177 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eb0ac6-cc9d-4fcb-a459-eb6810786b14 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:25.310569 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eb0ac6-cc9d-4fcb-a459-eb6810786b14 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:25.321431 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef813d5d-2800-46a7-947e-a7dc56655c8e None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:25.361761 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd8c55-2aeb-4e4f-80a7-dcbc60819149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:25.362544 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd8c55-2aeb-4e4f-80a7-dcbc60819149 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:25.362907 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd8c55-2aeb-4e4f-80a7-dcbc60819149 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:25.363309 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fd8c55-2aeb-4e4f-80a7-dcbc60819149 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:25.364128 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd8c55-2aeb-4e4f-80a7-dcbc60819149 None admin] RBAC: Adding query filter params (role.id=69610ceae19c4a2b963433e2a9f5bb26, scope.project.id=a10f1afbb5964a199da6c71dddab1bfd, user.id=d158c81dd66f4f0181f82eb165a7b8a0) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:25.364515 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd8c55-2aeb-4e4f-80a7-dcbc60819149 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:25.366384 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fd8c55-2aeb-4e4f-80a7-dcbc60819149 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:25.375510 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 66/126] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Oct 23 19:52:25 2022] GET /identity/v3/role_assignments?user.id=d158c81dd66f4f0181f82eb165a7b8a0&scope.project.id=a10f1afbb5964a199da6c71dddab1bfd&role.id=69610ceae19c4a2b963433e2a9f5bb26 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:25.434387 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 61/127] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Oct 23 19:52:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 19:52:25.440798 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd8c55-2aeb-4e4f-80a7-dcbc60819149 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:25.464731 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:25.567943 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fd8c55-2aeb-4e4f-80a7-dcbc60819149 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:25.608111 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725e5aba-d3c2-4251-95cb-d30c0180ccb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:25.608758 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e5aba-d3c2-4251-95cb-d30c0180ccb3 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:25.609040 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e5aba-d3c2-4251-95cb-d30c0180ccb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:25.609832 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725e5aba-d3c2-4251-95cb-d30c0180ccb3 None admin] PATH_INFO: `/v3/projects/0a195ad6e19d43ba991afe5458887d8c` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:25.610996 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725e5aba-d3c2-4251-95cb-d30c0180ccb3 None admin] RBAC: Authorizing `identity:get_project(project_id=0a195ad6e19d43ba991afe5458887d8c)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:25.612463 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725e5aba-d3c2-4251-95cb-d30c0180ccb3 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:25.614223 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 67/128] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Oct 23 19:52:25 2022] GET /identity/v3/projects/0a195ad6e19d43ba991afe5458887d8c => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:25.620154 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb0ac6-cc9d-4fcb-a459-eb6810786b14 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:25.705328 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:25.729394 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb0ac6-cc9d-4fcb-a459-eb6810786b14 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:25.764893 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78661b60-4615-4053-9c7f-cb5fde69c613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:25.765514 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78661b60-4615-4053-9c7f-cb5fde69c613 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:25.766536 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78661b60-4615-4053-9c7f-cb5fde69c613 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:25.766927 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78661b60-4615-4053-9c7f-cb5fde69c613 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:25.767751 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78661b60-4615-4053-9c7f-cb5fde69c613 None admin] RBAC: Adding query filter params (role.id=2989d29f392d4d8c802354159891eb11, scope.project.id=0a195ad6e19d43ba991afe5458887d8c, user.id=d158c81dd66f4f0181f82eb165a7b8a0) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:25.768218 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78661b60-4615-4053-9c7f-cb5fde69c613 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:25.770093 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78661b60-4615-4053-9c7f-cb5fde69c613 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:25.782319 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 62/129] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Oct 23 19:52:25 2022] GET /identity/v3/role_assignments?user.id=d158c81dd66f4f0181f82eb165a7b8a0&scope.project.id=0a195ad6e19d43ba991afe5458887d8c&role.id=2989d29f392d4d8c802354159891eb11 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:27.193064 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0513399-414a-424c-b2cf-0df2d8a8a78c None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:27.193568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0513399-414a-424c-b2cf-0df2d8a8a78c None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:27.193647 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0513399-414a-424c-b2cf-0df2d8a8a78c None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:27.194991 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 68/130] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:27.200304 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0900b-a095-49e2-986d-8b1ea866914c None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:27.200700 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0900b-a095-49e2-986d-8b1ea866914c None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:27.200849 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b0900b-a095-49e2-986d-8b1ea866914c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:27.219758 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-48b0900b-a095-49e2-986d-8b1ea866914c None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:27.220968 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-48b0900b-a095-49e2-986d-8b1ea866914c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:27.268006 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c762d-ad54-45d5-ac5e-30e0f43993cd None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:27.268291 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c762d-ad54-45d5-ac5e-30e0f43993cd None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:27.268576 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98c762d-ad54-45d5-ac5e-30e0f43993cd None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:27.269914 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 69/131] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 19:52:27.275074 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee942ca-c511-459d-a8db-60d4616d2c60 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:27.275311 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee942ca-c511-459d-a8db-60d4616d2c60 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:27.275774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee942ca-c511-459d-a8db-60d4616d2c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:27.286860 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 63/132] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:27.293953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-aee942ca-c511-459d-a8db-60d4616d2c60 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:27.295199 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-aee942ca-c511-459d-a8db-60d4616d2c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:27.360892 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 70/133] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 19:52:27.553953 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b0900b-a095-49e2-986d-8b1ea866914c None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:27.620619 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:27.630538 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee942ca-c511-459d-a8db-60d4616d2c60 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:27.641859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:27.653353 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b0900b-a095-49e2-986d-8b1ea866914c None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:27.687151 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412348d-de5a-44be-a4ba-e422dc7e0033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:27.688335 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412348d-de5a-44be-a4ba-e422dc7e0033 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:27.688618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412348d-de5a-44be-a4ba-e422dc7e0033 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:27.688897 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a412348d-de5a-44be-a4ba-e422dc7e0033 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:27.690001 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a412348d-de5a-44be-a4ba-e422dc7e0033 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:27.691541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a412348d-de5a-44be-a4ba-e422dc7e0033 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:27.693134 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 64/134] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:52:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:27.698502 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412348d-de5a-44be-a4ba-e422dc7e0033 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:27.729096 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee942ca-c511-459d-a8db-60d4616d2c60 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:27.763246 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2370c-2490-4b67-9b25-7f81ed3c74ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:27.763880 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2370c-2490-4b67-9b25-7f81ed3c74ee None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:27.764161 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2370c-2490-4b67-9b25-7f81ed3c74ee None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:27.764436 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2370c-2490-4b67-9b25-7f81ed3c74ee None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:27.774720 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c2370c-2490-4b67-9b25-7f81ed3c74ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:27.779555 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-73c2370c-2490-4b67-9b25-7f81ed3c74ee None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 19:52:27.779555 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:27.779555 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:27.779555 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:27.779555 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:27.779555 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:27.779555 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:27.779555 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:27.779555 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:27.779555 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:27.779555 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:27.779555 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:27.779555 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:27.779555 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:27.779555 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:27.779555 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:27.780392 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:27.780392 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:27.780392 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:27.780392 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:27.780392 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:27.780392 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:27.780392 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:27.780392 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:27.780392 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:27.780392 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:27.780392 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:27.780392 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:27.780392 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:27.780392 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:27.780392 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:27.780392 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:27.781138 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:27.781138 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:27.781138 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:27.781138 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:27.781138 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:27.781138 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:27.781138 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:27.781138 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:27.781138 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:27.781138 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:27.781138 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 19:52:27.781138 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:52:27.781138 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 71/135] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:52:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:52:27.787943 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2370c-2490-4b67-9b25-7f81ed3c74ee None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:27.793979 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a412348d-de5a-44be-a4ba-e422dc7e0033 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:27.830263 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498d9fa-e4e7-4221-a3b4-b5cfc7e5d5dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:27.830879 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498d9fa-e4e7-4221-a3b4-b5cfc7e5d5dc None admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:27.831168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498d9fa-e4e7-4221-a3b4-b5cfc7e5d5dc None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:27.831446 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498d9fa-e4e7-4221-a3b4-b5cfc7e5d5dc None admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:27.832104 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498d9fa-e4e7-4221-a3b4-b5cfc7e5d5dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:27.833540 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498d9fa-e4e7-4221-a3b4-b5cfc7e5d5dc None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:27.844478 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:27.855773 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 65/136] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Oct 23 19:52:27 2022] POST /identity/v3/users => generated 283 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 19:52:27.867261 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:27.892285 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2370c-2490-4b67-9b25-7f81ed3c74ee None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:27.929328 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b41f46-a0aa-49d0-86d4-0d16315d246e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:27.929955 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b41f46-a0aa-49d0-86d4-0d16315d246e None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:27.930238 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b41f46-a0aa-49d0-86d4-0d16315d246e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:27.930542 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b41f46-a0aa-49d0-86d4-0d16315d246e None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:27.931230 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b41f46-a0aa-49d0-86d4-0d16315d246e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:27.931531 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b41f46-a0aa-49d0-86d4-0d16315d246e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:27.932945 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b41f46-a0aa-49d0-86d4-0d16315d246e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:27.942273 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 72/137] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Oct 23 19:52:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:27.948350 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498d9fa-e4e7-4221-a3b4-b5cfc7e5d5dc None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:28.054761 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498d9fa-e4e7-4221-a3b4-b5cfc7e5d5dc None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:28.088585 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c4c00-22c6-4046-9ebb-30d06720a095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:28.089148 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c4c00-22c6-4046-9ebb-30d06720a095 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:28.089429 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c4c00-22c6-4046-9ebb-30d06720a095 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:28.089701 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c4c00-22c6-4046-9ebb-30d06720a095 None admin] PATH_INFO: `/v3/users/c569ecb5d4b2433abf6c464f88cf2ec1` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:28.090853 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c4c00-22c6-4046-9ebb-30d06720a095 None admin] RBAC: Authorizing `identity:get_user(user_id=c569ecb5d4b2433abf6c464f88cf2ec1)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:28.092294 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c4c00-22c6-4046-9ebb-30d06720a095 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:28.094469 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 66/138] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:52:27 2022] GET /identity/v3/users/c569ecb5d4b2433abf6c464f88cf2ec1 => generated 289 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:28.100146 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b41f46-a0aa-49d0-86d4-0d16315d246e None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:28.203609 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b41f46-a0aa-49d0-86d4-0d16315d246e None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:28.236530 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:28.242362 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58135c7-815c-4912-b01c-ba1a89742565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:28.242987 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58135c7-815c-4912-b01c-ba1a89742565 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:28.243294 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58135c7-815c-4912-b01c-ba1a89742565 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:28.245232 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58135c7-815c-4912-b01c-ba1a89742565 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:28.325026 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58135c7-815c-4912-b01c-ba1a89742565 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:28.326533 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58135c7-815c-4912-b01c-ba1a89742565 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:28.434140 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 73/139] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Oct 23 19:52:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 19:52:28.440237 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c4c00-22c6-4046-9ebb-30d06720a095 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:28.506692 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:28.550332 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c4c00-22c6-4046-9ebb-30d06720a095 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:28.585957 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f61a1e1-11e2-466f-a2ff-11430c145b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:28.586910 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61a1e1-11e2-466f-a2ff-11430c145b99 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:28.587412 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61a1e1-11e2-466f-a2ff-11430c145b99 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:28.587858 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f61a1e1-11e2-466f-a2ff-11430c145b99 None admin] PATH_INFO: `/v3/projects/0a195ad6e19d43ba991afe5458887d8c` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:28.589190 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f61a1e1-11e2-466f-a2ff-11430c145b99 None admin] RBAC: Authorizing `identity:get_project(project_id=0a195ad6e19d43ba991afe5458887d8c)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:28.590912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f61a1e1-11e2-466f-a2ff-11430c145b99 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:28.593085 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 67/140] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Oct 23 19:52:28 2022] GET /identity/v3/projects/0a195ad6e19d43ba991afe5458887d8c => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:28.599142 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58135c7-815c-4912-b01c-ba1a89742565 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:28.632798 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:28.712665 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58135c7-815c-4912-b01c-ba1a89742565 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:28.748648 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206d4110-a4f7-4578-9ea8-c7ef4a516155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:28.749544 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d4110-a4f7-4578-9ea8-c7ef4a516155 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:28.750048 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d4110-a4f7-4578-9ea8-c7ef4a516155 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:28.750522 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d4110-a4f7-4578-9ea8-c7ef4a516155 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:28.751447 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206d4110-a4f7-4578-9ea8-c7ef4a516155 None admin] RBAC: Adding query filter params (role.id=54768ecdabbd443ca182db572b13b3ca, scope.project.id=0a195ad6e19d43ba991afe5458887d8c, user.id=c569ecb5d4b2433abf6c464f88cf2ec1) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:28.751925 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206d4110-a4f7-4578-9ea8-c7ef4a516155 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:28.753530 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206d4110-a4f7-4578-9ea8-c7ef4a516155 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:28.759688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 74/141] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Oct 23 19:52:28 2022] GET /identity/v3/role_assignments?user.id=c569ecb5d4b2433abf6c464f88cf2ec1&scope.project.id=0a195ad6e19d43ba991afe5458887d8c&role.id=54768ecdabbd443ca182db572b13b3ca => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:30.119593 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ff390-0285-4e8b-87a7-a49374e03282 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:30.120141 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ff390-0285-4e8b-87a7-a49374e03282 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:30.120141 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ff390-0285-4e8b-87a7-a49374e03282 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:30.121644 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 68/142] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:30.130290 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8cc938-983f-4f85-bb03-7652bf896964 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:30.130290 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8cc938-983f-4f85-bb03-7652bf896964 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:30.130290 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8cc938-983f-4f85-bb03-7652bf896964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:30.147441 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-ff8cc938-983f-4f85-bb03-7652bf896964 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:30.148629 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ff8cc938-983f-4f85-bb03-7652bf896964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:30.226845 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 75/143] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:30.294102 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ab5e1-d7ff-4637-8700-164afb8dbf1f None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:30.294102 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ab5e1-d7ff-4637-8700-164afb8dbf1f None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:30.294102 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ab5e1-d7ff-4637-8700-164afb8dbf1f None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:30.294883 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 69/144] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:30.300437 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517f49b-540c-48de-8a48-2e46d6c81c98 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:30.300729 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517f49b-540c-48de-8a48-2e46d6c81c98 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:30.301009 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517f49b-540c-48de-8a48-2e46d6c81c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:30.319429 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-6517f49b-540c-48de-8a48-2e46d6c81c98 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:30.320619 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-6517f49b-540c-48de-8a48-2e46d6c81c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:30.343124 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:30.361931 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6119e-5cfb-40d4-86f9-d596d9f4f754 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:30.362391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6119e-5cfb-40d4-86f9-d596d9f4f754 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:30.362763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6119e-5cfb-40d4-86f9-d596d9f4f754 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:30.364386 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 70/145] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 19:52:30.369435 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54f74f-e44e-45b1-8e0d-f523b531161c None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:30.369797 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54f74f-e44e-45b1-8e0d-f523b531161c None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:30.370142 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a54f74f-e44e-45b1-8e0d-f523b531161c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:30.389460 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-2a54f74f-e44e-45b1-8e0d-f523b531161c None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:30.391302 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-2a54f74f-e44e-45b1-8e0d-f523b531161c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:30.402521 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 76/146] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:30.466828 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 71/147] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:30.524250 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6517f49b-540c-48de-8a48-2e46d6c81c98 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:30.623165 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6517f49b-540c-48de-8a48-2e46d6c81c98 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:30.655908 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f391185-6a37-40c0-b5e7-e474db967255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:30.657108 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f391185-6a37-40c0-b5e7-e474db967255 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:30.657391 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f391185-6a37-40c0-b5e7-e474db967255 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:30.657664 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f391185-6a37-40c0-b5e7-e474db967255 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:30.667658 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f391185-6a37-40c0-b5e7-e474db967255 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:30.672173 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a54f74f-e44e-45b1-8e0d-f523b531161c None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:30.672766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-3f391185-6a37-40c0-b5e7-e474db967255 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:52:30.672766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:30.672766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:30.672766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:30.672766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:30.672766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:30.672766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:30.672766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:30.672766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:30.672766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:30.672766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:30.672766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:30.672766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:30.672766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:30.672766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:30.672766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:30.673614 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:30.673614 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:30.673614 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:30.673614 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:30.673614 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:30.673614 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:30.673614 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:30.673614 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:30.673614 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:30.673614 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:30.673614 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:30.673614 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:30.673614 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:30.673614 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:30.673614 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:30.673614 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:30.674451 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:30.674451 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:30.674451 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:30.674451 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:30.674451 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:30.674451 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:30.674451 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:30.674451 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:30.674451 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:30.674451 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:30.674451 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:52:30.674451 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:52:30.674451 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 77/148] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Oct 23 19:52:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:52:30.681061 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f391185-6a37-40c0-b5e7-e474db967255 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:30.699862 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:30.739403 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:30.779106 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a54f74f-e44e-45b1-8e0d-f523b531161c None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:30.786929 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f391185-6a37-40c0-b5e7-e474db967255 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:30.812093 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e47289-ee5b-48f5-9515-d9bdff412d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:30.813296 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e47289-ee5b-48f5-9515-d9bdff412d97 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:30.813580 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e47289-ee5b-48f5-9515-d9bdff412d97 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:30.813856 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e47289-ee5b-48f5-9515-d9bdff412d97 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:30.822059 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a50ae-b3be-4ece-b6d9-4675ea021e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:30.822969 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a50ae-b3be-4ece-b6d9-4675ea021e8e None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:30.823253 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a50ae-b3be-4ece-b6d9-4675ea021e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:30.823528 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6a50ae-b3be-4ece-b6d9-4675ea021e8e None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:30.823687 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e47289-ee5b-48f5-9515-d9bdff412d97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:30.824195 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6a50ae-b3be-4ece-b6d9-4675ea021e8e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:30.824500 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6a50ae-b3be-4ece-b6d9-4675ea021e8e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:30.825866 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6a50ae-b3be-4ece-b6d9-4675ea021e8e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:30.828618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-a2e47289-ee5b-48f5-9515-d9bdff412d97 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 19:52:30.828618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:30.828618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:30.828618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:30.828618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:30.828618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:30.828618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:30.828618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:30.828618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:30.828618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:30.828618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:30.828618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:30.828618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:30.828618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:30.828618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:30.828618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:30.829490 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:30.829490 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:30.829490 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:30.829490 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:30.829490 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:30.829490 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:30.829490 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:30.829490 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:30.829490 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:30.829490 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:30.829490 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:30.829490 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:30.829490 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:30.829490 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:30.829490 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:30.829490 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:30.830291 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:30.830291 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:30.830291 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:30.830291 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:30.830291 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:30.830291 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:30.830291 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:30.830291 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:30.830291 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:30.830291 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:30.830291 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 19:52:30.830291 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:52:30.830291 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 72/149] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Oct 23 19:52:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:52:30.834690 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 78/150] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:52:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:30.835076 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e47289-ee5b-48f5-9515-d9bdff412d97 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:30.839914 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a50ae-b3be-4ece-b6d9-4675ea021e8e None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:30.917990 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:30.933987 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e47289-ee5b-48f5-9515-d9bdff412d97 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:30.940160 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6a50ae-b3be-4ece-b6d9-4675ea021e8e None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:30.956715 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:30.967897 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47521f9f-6ca2-4de2-ad35-bb0fdc6941e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:30.968514 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47521f9f-6ca2-4de2-ad35-bb0fdc6941e8 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:30.969369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47521f9f-6ca2-4de2-ad35-bb0fdc6941e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:30.969643 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47521f9f-6ca2-4de2-ad35-bb0fdc6941e8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:30.970776 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47521f9f-6ca2-4de2-ad35-bb0fdc6941e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:30.972211 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47521f9f-6ca2-4de2-ad35-bb0fdc6941e8 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:30.973929 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 73/151] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:52:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:30.974030 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1abb526-56e4-4e0d-922a-aa7a75383325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:30.974512 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1abb526-56e4-4e0d-922a-aa7a75383325 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:30.974620 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47521f9f-6ca2-4de2-ad35-bb0fdc6941e8 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:30.974849 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1abb526-56e4-4e0d-922a-aa7a75383325 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:30.975138 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1abb526-56e4-4e0d-922a-aa7a75383325 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:30.975815 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1abb526-56e4-4e0d-922a-aa7a75383325 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:30.976122 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1abb526-56e4-4e0d-922a-aa7a75383325 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:30.977695 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1abb526-56e4-4e0d-922a-aa7a75383325 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:30.990524 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 79/152] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:52:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:30.990524 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1abb526-56e4-4e0d-922a-aa7a75383325 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:31.075496 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47521f9f-6ca2-4de2-ad35-bb0fdc6941e8 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:31.090958 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1abb526-56e4-4e0d-922a-aa7a75383325 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:31.108801 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f8052-2179-4ea7-9b18-64136523ae0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:31.109957 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f8052-2179-4ea7-9b18-64136523ae0f None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:31.110236 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f8052-2179-4ea7-9b18-64136523ae0f None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:31.110533 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f8052-2179-4ea7-9b18-64136523ae0f None admin] PATH_INFO: `/v3/users/e5b99caa28074137863d87b3756ea9cc` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:31.123412 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843f8052-2179-4ea7-9b18-64136523ae0f None admin] RBAC: Authorizing `identity:get_user(user_id=e5b99caa28074137863d87b3756ea9cc)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:31.123945 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320d439-e2d4-4518-8125-ac31ea7cb4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:31.124416 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320d439-e2d4-4518-8125-ac31ea7cb4fe None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:31.124698 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320d439-e2d4-4518-8125-ac31ea7cb4fe None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:31.124852 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843f8052-2179-4ea7-9b18-64136523ae0f None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:31.125030 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320d439-e2d4-4518-8125-ac31ea7cb4fe None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:31.126475 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5320d439-e2d4-4518-8125-ac31ea7cb4fe None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:31.127045 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 74/153] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:52:30 2022] GET /identity/v3/users/e5b99caa28074137863d87b3756ea9cc => generated 283 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:31.127815 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f8052-2179-4ea7-9b18-64136523ae0f None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:31.131158 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5320d439-e2d4-4518-8125-ac31ea7cb4fe None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:31.142476 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:31.153383 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 80/154] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Oct 23 19:52:30 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:52:31.154288 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320d439-e2d4-4518-8125-ac31ea7cb4fe None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:31.192013 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:31.249517 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f8052-2179-4ea7-9b18-64136523ae0f None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:31.270710 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320d439-e2d4-4518-8125-ac31ea7cb4fe None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:31.286378 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89068f64-e9e8-4254-b03a-9ab86b1e5644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:31.287294 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89068f64-e9e8-4254-b03a-9ab86b1e5644 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:31.287577 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89068f64-e9e8-4254-b03a-9ab86b1e5644 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:31.287857 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89068f64-e9e8-4254-b03a-9ab86b1e5644 None admin] PATH_INFO: `/v3/users/e5b99caa28074137863d87b3756ea9cc` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:31.289407 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89068f64-e9e8-4254-b03a-9ab86b1e5644 None admin] RBAC: Authorizing `identity:get_user(user_id=e5b99caa28074137863d87b3756ea9cc)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:31.291448 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89068f64-e9e8-4254-b03a-9ab86b1e5644 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:31.294400 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 75/155] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:52:31 2022] GET /identity/v3/users/e5b99caa28074137863d87b3756ea9cc => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:31.299498 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89068f64-e9e8-4254-b03a-9ab86b1e5644 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:31.314633 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b11249-531c-41db-a837-24c46eee14f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:31.315332 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b11249-531c-41db-a837-24c46eee14f4 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:31.315618 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b11249-531c-41db-a837-24c46eee14f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:31.315944 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b11249-531c-41db-a837-24c46eee14f4 None admin] PATH_INFO: `/v3/projects/6da23d78618243288286bfd8d4b7c9d9` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:31.317055 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b11249-531c-41db-a837-24c46eee14f4 None admin] RBAC: Authorizing `identity:get_project(project_id=6da23d78618243288286bfd8d4b7c9d9)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:31.318592 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b11249-531c-41db-a837-24c46eee14f4 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:31.320438 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 81/156] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Oct 23 19:52:31 2022] GET /identity/v3/projects/6da23d78618243288286bfd8d4b7c9d9 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:31.326927 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b11249-531c-41db-a837-24c46eee14f4 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:31.399695 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:31.426384 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89068f64-e9e8-4254-b03a-9ab86b1e5644 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:31.448129 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b11249-531c-41db-a837-24c46eee14f4 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:31.448881 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:31.471052 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4f782-d79e-4838-80be-8e3225012809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:31.472573 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4f782-d79e-4838-80be-8e3225012809 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:31.473499 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4f782-d79e-4838-80be-8e3225012809 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:31.474017 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4f782-d79e-4838-80be-8e3225012809 None admin] PATH_INFO: `/v3/projects/6da23d78618243288286bfd8d4b7c9d9` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:31.476150 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4f782-d79e-4838-80be-8e3225012809 None admin] RBAC: Authorizing `identity:get_project(project_id=6da23d78618243288286bfd8d4b7c9d9)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:31.478007 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae4f782-d79e-4838-80be-8e3225012809 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:31.479956 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 76/157] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Oct 23 19:52:31 2022] GET /identity/v3/projects/6da23d78618243288286bfd8d4b7c9d9 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:31.485299 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4f782-d79e-4838-80be-8e3225012809 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:31.487035 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b7181a-62f7-411d-85b5-b0ba08e80cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:31.488307 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7181a-62f7-411d-85b5-b0ba08e80cc7 None admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:31.488307 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7181a-62f7-411d-85b5-b0ba08e80cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:31.488307 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7181a-62f7-411d-85b5-b0ba08e80cc7 None admin] PATH_INFO: `/v3/projects/6da23d78618243288286bfd8d4b7c9d9/users/e5b99caa28074137863d87b3756ea9cc/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:31.490320 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b7181a-62f7-411d-85b5-b0ba08e80cc7 None admin] RBAC: Authorizing `identity:create_grant(project_id=6da23d78618243288286bfd8d4b7c9d9, user_id=e5b99caa28074137863d87b3756ea9cc, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:31.491773 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b7181a-62f7-411d-85b5-b0ba08e80cc7 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:31.512447 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 82/158] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Oct 23 19:52:31 2022] PUT /identity/v3/projects/6da23d78618243288286bfd8d4b7c9d9/users/e5b99caa28074137863d87b3756ea9cc/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 19:52:31.614992 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4f782-d79e-4838-80be-8e3225012809 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:31.654478 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ceb1470-f727-4017-8cca-5014ce241522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:31.655163 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb1470-f727-4017-8cca-5014ce241522 None admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:31.655554 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb1470-f727-4017-8cca-5014ce241522 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:31.655837 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ceb1470-f727-4017-8cca-5014ce241522 None admin] PATH_INFO: `/v3/projects/6da23d78618243288286bfd8d4b7c9d9/users/e5b99caa28074137863d87b3756ea9cc/roles/2989d29f392d4d8c802354159891eb11` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:31.658330 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceb1470-f727-4017-8cca-5014ce241522 None admin] RBAC: Authorizing `identity:create_grant(project_id=6da23d78618243288286bfd8d4b7c9d9, user_id=e5b99caa28074137863d87b3756ea9cc, role_id=2989d29f392d4d8c802354159891eb11)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:31.659842 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ceb1470-f727-4017-8cca-5014ce241522 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:31.677602 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 77/159] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Oct 23 19:52:31 2022] PUT /identity/v3/projects/6da23d78618243288286bfd8d4b7c9d9/users/e5b99caa28074137863d87b3756ea9cc/roles/2989d29f392d4d8c802354159891eb11 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 19:52:33.481940 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02f2df-3912-4424-a565-d03833e8f235 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:33.483003 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02f2df-3912-4424-a565-d03833e8f235 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:33.483490 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02f2df-3912-4424-a565-d03833e8f235 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:33.485889 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 83/160] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:33.491385 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07afd43-b757-491d-81d7-07cff145cb3a None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:33.492189 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07afd43-b757-491d-81d7-07cff145cb3a None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:33.492656 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07afd43-b757-491d-81d7-07cff145cb3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:33.511611 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-d07afd43-b757-491d-81d7-07cff145cb3a None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:33.513140 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-d07afd43-b757-491d-81d7-07cff145cb3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:33.540244 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0aac9b-7114-44e5-b413-9c105e4bb1be None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:33.541012 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0aac9b-7114-44e5-b413-9c105e4bb1be None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:33.541490 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0aac9b-7114-44e5-b413-9c105e4bb1be None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:33.543943 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 84/161] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:33.550015 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b8481-394d-4d5d-8723-da23e8cbf0b1 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:33.550567 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b8481-394d-4d5d-8723-da23e8cbf0b1 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:33.551044 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43b8481-394d-4d5d-8723-da23e8cbf0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:33.557954 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:33.569933 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-f43b8481-394d-4d5d-8723-da23e8cbf0b1 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:33.571439 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f43b8481-394d-4d5d-8723-da23e8cbf0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:33.592892 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 78/162] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:33.647483 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 85/163] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 19:52:33.790929 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d59eb0-4d40-4373-b817-070dd433a8f2 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:33.791582 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d59eb0-4d40-4373-b817-070dd433a8f2 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:33.792065 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d59eb0-4d40-4373-b817-070dd433a8f2 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:33.793880 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 79/164] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:33.799512 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d7e5d-dd7c-4d05-b554-e219fb8d5a81 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:33.799803 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d7e5d-dd7c-4d05-b554-e219fb8d5a81 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:33.800083 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941d7e5d-dd7c-4d05-b554-e219fb8d5a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:33.819631 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-941d7e5d-dd7c-4d05-b554-e219fb8d5a81 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:33.821430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-941d7e5d-dd7c-4d05-b554-e219fb8d5a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:33.863454 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46265767-afb2-4399-866b-6330b642aa7d None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:33.863755 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46265767-afb2-4399-866b-6330b642aa7d None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:33.863999 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46265767-afb2-4399-866b-6330b642aa7d None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:33.865523 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 80/165] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 19:52:33.868571 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46265767-afb2-4399-866b-6330b642aa7d None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:33.879971 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:33.901304 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 86/166] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:33.908102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da2b2d-58f8-4aa6-9587-da177457d4af None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:33.908581 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da2b2d-58f8-4aa6-9587-da177457d4af None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:33.909025 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31da2b2d-58f8-4aa6-9587-da177457d4af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:33.931967 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-31da2b2d-58f8-4aa6-9587-da177457d4af None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:33.933753 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-31da2b2d-58f8-4aa6-9587-da177457d4af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:33.985145 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46265767-afb2-4399-866b-6330b642aa7d None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:34.002589 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 87/167] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:34.003414 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31da2b2d-58f8-4aa6-9587-da177457d4af None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:34.032550 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a34588c-1d3d-4ac9-aa4b-ee001984daa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:34.033189 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34588c-1d3d-4ac9-aa4b-ee001984daa3 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:34.033464 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34588c-1d3d-4ac9-aa4b-ee001984daa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:34.033735 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34588c-1d3d-4ac9-aa4b-ee001984daa3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:34.046144 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a34588c-1d3d-4ac9-aa4b-ee001984daa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:34.052259 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-1a34588c-1d3d-4ac9-aa4b-ee001984daa3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 19:52:34.052259 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:34.052259 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:34.052259 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:34.052259 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:34.052259 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:34.052259 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:34.052259 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:34.052259 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:34.052259 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:34.052259 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:34.052259 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:34.052259 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:34.052259 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:34.052259 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:34.052259 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:34.053194 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:34.053194 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:34.053194 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:34.053194 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:34.053194 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:34.053194 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:34.053194 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:34.053194 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:34.053194 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:34.053194 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:34.053194 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:34.053194 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:34.053194 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:34.053194 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:34.053194 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:34.053194 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:34.053964 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:34.053964 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:34.053964 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:34.053964 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:34.053964 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:34.053964 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:34.053964 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:34.053964 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:34.053964 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:34.053964 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:34.053964 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 19:52:34.053964 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:52:34.059769 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 81/168] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:52:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:52:34.063494 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a34588c-1d3d-4ac9-aa4b-ee001984daa3 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:34.100436 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:34.129592 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31da2b2d-58f8-4aa6-9587-da177457d4af None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:34.145196 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:34.170605 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649708e-a2ce-4f0c-ba08-c3a94f03e131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:34.171241 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649708e-a2ce-4f0c-ba08-c3a94f03e131 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:34.171517 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649708e-a2ce-4f0c-ba08-c3a94f03e131 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:34.171846 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3649708e-a2ce-4f0c-ba08-c3a94f03e131 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:34.173440 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3649708e-a2ce-4f0c-ba08-c3a94f03e131 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:34.174994 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3649708e-a2ce-4f0c-ba08-c3a94f03e131 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:34.176706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 88/169] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:52:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:34.177321 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649708e-a2ce-4f0c-ba08-c3a94f03e131 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:34.178182 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a34588c-1d3d-4ac9-aa4b-ee001984daa3 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:34.215212 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc6581-a682-4ebc-8ef7-ea0a2b3adb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:34.216163 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc6581-a682-4ebc-8ef7-ea0a2b3adb43 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:34.216634 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc6581-a682-4ebc-8ef7-ea0a2b3adb43 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:34.217220 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc6581-a682-4ebc-8ef7-ea0a2b3adb43 None admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:34.218001 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfc6581-a682-4ebc-8ef7-ea0a2b3adb43 None admin] RBAC: Adding query filter params (name=member) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:34.218388 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfc6581-a682-4ebc-8ef7-ea0a2b3adb43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:34.220476 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfc6581-a682-4ebc-8ef7-ea0a2b3adb43 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:34.231242 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 82/170] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Oct 23 19:52:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:34.232343 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc6581-a682-4ebc-8ef7-ea0a2b3adb43 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:34.281633 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3649708e-a2ce-4f0c-ba08-c3a94f03e131 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:34.316151 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95147c84-d805-4e23-99df-53ccbd9d6798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:34.316708 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95147c84-d805-4e23-99df-53ccbd9d6798 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:34.316988 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95147c84-d805-4e23-99df-53ccbd9d6798 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:34.317261 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95147c84-d805-4e23-99df-53ccbd9d6798 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:34.327232 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95147c84-d805-4e23-99df-53ccbd9d6798 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:34.327606 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:34.332123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-95147c84-d805-4e23-99df-53ccbd9d6798 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:52:34.332123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:34.332123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:34.332123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:34.332123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:34.332123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:34.332123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:34.332123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:34.332123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:34.332123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:34.332123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:34.332123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:34.332123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:34.332123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:34.332123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:34.332123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:34.332976 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:34.332976 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:34.332976 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:34.332976 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:34.332976 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:34.332976 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:34.332976 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:34.332976 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:34.332976 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:34.332976 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:34.332976 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:34.332976 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:34.332976 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:34.332976 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:34.332976 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:34.332976 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:34.333738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:34.333738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:34.333738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:34.333738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:34.333738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:34.333738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:34.333738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:34.333738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:34.333738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:34.333738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:34.333738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:52:34.333738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:52:34.333738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 89/171] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Oct 23 19:52:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:52:34.334441 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95147c84-d805-4e23-99df-53ccbd9d6798 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:34.334515 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc6581-a682-4ebc-8ef7-ea0a2b3adb43 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:34.367348 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87dc5fc-44f8-4261-8506-6343947a13d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:34.367945 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87dc5fc-44f8-4261-8506-6343947a13d8 None admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:34.368221 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87dc5fc-44f8-4261-8506-6343947a13d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:34.368489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87dc5fc-44f8-4261-8506-6343947a13d8 None admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:34.369135 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87dc5fc-44f8-4261-8506-6343947a13d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:34.370595 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87dc5fc-44f8-4261-8506-6343947a13d8 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:34.378182 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:34.391584 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 83/172] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Oct 23 19:52:34 2022] POST /identity/v3/users => generated 297 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:52:34.392394 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87dc5fc-44f8-4261-8506-6343947a13d8 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:34.464688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95147c84-d805-4e23-99df-53ccbd9d6798 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:34.504102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae192bb2-ba0d-4f80-b674-b75eaed6b0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:34.504792 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae192bb2-ba0d-4f80-b674-b75eaed6b0b8 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:34.505240 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae192bb2-ba0d-4f80-b674-b75eaed6b0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:34.505343 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae192bb2-ba0d-4f80-b674-b75eaed6b0b8 None admin] PATH_INFO: `/v3/users/cc2328fef2a347dc855f2823bcca687b` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:34.508049 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87dc5fc-44f8-4261-8506-6343947a13d8 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:34.522355 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae192bb2-ba0d-4f80-b674-b75eaed6b0b8 None admin] RBAC: Authorizing `identity:get_user(user_id=cc2328fef2a347dc855f2823bcca687b)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:34.523819 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae192bb2-ba0d-4f80-b674-b75eaed6b0b8 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:34.525631 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 90/173] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:52:34 2022] GET /identity/v3/users/cc2328fef2a347dc855f2823bcca687b => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:34.527132 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae192bb2-ba0d-4f80-b674-b75eaed6b0b8 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:34.547313 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100447b-6f08-4f65-b22c-6267e4aa786c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:34.547974 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100447b-6f08-4f65-b22c-6267e4aa786c None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:34.548290 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100447b-6f08-4f65-b22c-6267e4aa786c None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:34.548672 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7100447b-6f08-4f65-b22c-6267e4aa786c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:34.560650 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7100447b-6f08-4f65-b22c-6267e4aa786c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:34.566832 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-7100447b-6f08-4f65-b22c-6267e4aa786c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 19:52:34.566832 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:34.566832 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:34.566832 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:34.566832 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:34.566832 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:34.566832 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:34.566832 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:34.566832 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:34.566832 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:34.566832 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:34.566832 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:34.566832 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:34.566832 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:34.566832 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:34.566832 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:34.567703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:34.567703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:34.567703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:34.567703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:34.567703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:34.567703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:34.567703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:34.567703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:34.567703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:34.567703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:34.567703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:34.567703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:34.567703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:34.567703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:34.567703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:34.567703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:34.568489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:34.568489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:34.568489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:34.568489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:34.568489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:34.568489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:34.568489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:34.568489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:34.568489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:34.568489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:34.568489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 19:52:34.568489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:52:34.568489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 84/174] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Oct 23 19:52:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:52:34.569118 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100447b-6f08-4f65-b22c-6267e4aa786c None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:34.590140 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:34.644720 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:34.647658 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae192bb2-ba0d-4f80-b674-b75eaed6b0b8 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:34.694588 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b797de5a-5ba5-476e-a952-7dbe915c4ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:34.695298 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797de5a-5ba5-476e-a952-7dbe915c4ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:34.695932 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797de5a-5ba5-476e-a952-7dbe915c4ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:34.695932 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797de5a-5ba5-476e-a952-7dbe915c4ec2 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:34.696509 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b797de5a-5ba5-476e-a952-7dbe915c4ec2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:34.696812 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b797de5a-5ba5-476e-a952-7dbe915c4ec2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:34.698203 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b797de5a-5ba5-476e-a952-7dbe915c4ec2 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:34.702236 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7100447b-6f08-4f65-b22c-6267e4aa786c None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:34.707888 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 91/175] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:52:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:34.713659 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b797de5a-5ba5-476e-a952-7dbe915c4ec2 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:34.744626 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c5c6b7-91df-4350-a57d-44b5562ebc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:34.745261 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5c6b7-91df-4350-a57d-44b5562ebc73 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:34.745536 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5c6b7-91df-4350-a57d-44b5562ebc73 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:34.745804 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c5c6b7-91df-4350-a57d-44b5562ebc73 None admin] PATH_INFO: `/v3/projects/6da23d78618243288286bfd8d4b7c9d9` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:34.746953 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c5c6b7-91df-4350-a57d-44b5562ebc73 None admin] RBAC: Authorizing `identity:get_project(project_id=6da23d78618243288286bfd8d4b7c9d9)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:34.748404 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c5c6b7-91df-4350-a57d-44b5562ebc73 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:34.750647 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 85/176] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Oct 23 19:52:34 2022] GET /identity/v3/projects/6da23d78618243288286bfd8d4b7c9d9 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:34.751406 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c5c6b7-91df-4350-a57d-44b5562ebc73 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:34.835767 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b797de5a-5ba5-476e-a952-7dbe915c4ec2 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:34.846614 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:34.870417 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c5c6b7-91df-4350-a57d-44b5562ebc73 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:34.876171 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a3340-0322-4293-a570-8ee6f5d01d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:34.876904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a3340-0322-4293-a570-8ee6f5d01d50 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:34.877288 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a3340-0322-4293-a570-8ee6f5d01d50 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:34.877682 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a3340-0322-4293-a570-8ee6f5d01d50 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:34.878487 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a3340-0322-4293-a570-8ee6f5d01d50 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:34.878893 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a3340-0322-4293-a570-8ee6f5d01d50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:34.880371 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a3340-0322-4293-a570-8ee6f5d01d50 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:34.889499 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 92/177] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:52:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:34.890486 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a3340-0322-4293-a570-8ee6f5d01d50 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:34.914291 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0814e-2ae7-46e3-85f7-a8d24c4b64a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:34.915570 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0814e-2ae7-46e3-85f7-a8d24c4b64a9 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:34.916246 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0814e-2ae7-46e3-85f7-a8d24c4b64a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:34.916664 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0814e-2ae7-46e3-85f7-a8d24c4b64a9 None admin] PATH_INFO: `/v3/users/e5b99caa28074137863d87b3756ea9cc` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:34.918140 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea0814e-2ae7-46e3-85f7-a8d24c4b64a9 None admin] RBAC: Authorizing `identity:get_user(user_id=e5b99caa28074137863d87b3756ea9cc)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:34.920325 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea0814e-2ae7-46e3-85f7-a8d24c4b64a9 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:34.923011 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 86/178] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:52:34 2022] GET /identity/v3/users/e5b99caa28074137863d87b3756ea9cc => generated 283 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:34.923911 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0814e-2ae7-46e3-85f7-a8d24c4b64a9 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:34.934643 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:35.007800 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a3340-0322-4293-a570-8ee6f5d01d50 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:35.044660 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0814e-2ae7-46e3-85f7-a8d24c4b64a9 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:35.046374 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d9d3dc-e6ec-49e1-99c7-ae22b9b2bea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:35.047324 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9d3dc-e6ec-49e1-99c7-ae22b9b2bea0 None admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:35.048512 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9d3dc-e6ec-49e1-99c7-ae22b9b2bea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:35.048876 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d9d3dc-e6ec-49e1-99c7-ae22b9b2bea0 None admin] PATH_INFO: `/v3/projects/6da23d78618243288286bfd8d4b7c9d9/users/cc2328fef2a347dc855f2823bcca687b/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:35.050931 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d9d3dc-e6ec-49e1-99c7-ae22b9b2bea0 None admin] RBAC: Authorizing `identity:create_grant(project_id=6da23d78618243288286bfd8d4b7c9d9, user_id=cc2328fef2a347dc855f2823bcca687b, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:35.052450 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d9d3dc-e6ec-49e1-99c7-ae22b9b2bea0 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:35.070049 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 93/179] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Oct 23 19:52:34 2022] PUT /identity/v3/projects/6da23d78618243288286bfd8d4b7c9d9/users/cc2328fef2a347dc855f2823bcca687b/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 19:52:35.070907 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d9d3dc-e6ec-49e1-99c7-ae22b9b2bea0 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:35.085280 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dab586-af71-4688-ac15-3f30f8fc167c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:35.094784 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dab586-af71-4688-ac15-3f30f8fc167c None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:35.095533 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dab586-af71-4688-ac15-3f30f8fc167c None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:35.096054 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dab586-af71-4688-ac15-3f30f8fc167c None admin] PATH_INFO: `/v3/users/e5b99caa28074137863d87b3756ea9cc` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:35.097720 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dab586-af71-4688-ac15-3f30f8fc167c None admin] RBAC: Authorizing `identity:get_user(user_id=e5b99caa28074137863d87b3756ea9cc)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:35.099509 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dab586-af71-4688-ac15-3f30f8fc167c None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:35.101835 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 87/180] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:52:34 2022] GET /identity/v3/users/e5b99caa28074137863d87b3756ea9cc => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:35.107584 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dab586-af71-4688-ac15-3f30f8fc167c None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:35.159089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:35.234634 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:35.260830 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d9d3dc-e6ec-49e1-99c7-ae22b9b2bea0 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:35.293127 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dab586-af71-4688-ac15-3f30f8fc167c None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:35.302444 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb120b1-4627-49b8-8888-e84f4e754d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:35.303068 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb120b1-4627-49b8-8888-e84f4e754d1c None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:35.303348 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb120b1-4627-49b8-8888-e84f4e754d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:35.303616 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb120b1-4627-49b8-8888-e84f4e754d1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:35.338620 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76616d2d-be24-4049-8406-831cfd8ad209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:35.339350 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76616d2d-be24-4049-8406-831cfd8ad209 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:35.339630 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76616d2d-be24-4049-8406-831cfd8ad209 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:35.339899 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76616d2d-be24-4049-8406-831cfd8ad209 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:35.398105 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb120b1-4627-49b8-8888-e84f4e754d1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:35.399694 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb120b1-4627-49b8-8888-e84f4e754d1c None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:35.413083 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76616d2d-be24-4049-8406-831cfd8ad209 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:35.414589 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76616d2d-be24-4049-8406-831cfd8ad209 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:35.434629 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:35.525064 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 88/181] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Oct 23 19:52:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 23 19:52:35.535405 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76616d2d-be24-4049-8406-831cfd8ad209 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:35.638971 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76616d2d-be24-4049-8406-831cfd8ad209 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:35.643810 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:35.662787 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:35.679303 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d76c2-4023-41f7-b09d-8222e9bcfc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:35.680032 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d76c2-4023-41f7-b09d-8222e9bcfc82 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:35.680315 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d76c2-4023-41f7-b09d-8222e9bcfc82 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:35.680587 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415d76c2-4023-41f7-b09d-8222e9bcfc82 None admin] PATH_INFO: `/v3/projects/6da23d78618243288286bfd8d4b7c9d9` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:35.681917 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415d76c2-4023-41f7-b09d-8222e9bcfc82 None admin] RBAC: Authorizing `identity:get_project(project_id=6da23d78618243288286bfd8d4b7c9d9)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:35.683431 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415d76c2-4023-41f7-b09d-8222e9bcfc82 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:35.688128 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 89/182] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Oct 23 19:52:35 2022] GET /identity/v3/projects/6da23d78618243288286bfd8d4b7c9d9 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:35.692597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 94/183] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Oct 23 19:52:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 622 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 19:52:35.693870 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d76c2-4023-41f7-b09d-8222e9bcfc82 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:35.699051 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb120b1-4627-49b8-8888-e84f4e754d1c None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:35.832316 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb120b1-4627-49b8-8888-e84f4e754d1c None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:35.878310 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d9784d-d70b-424e-ae43-a05e14c6ee19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:35.879209 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9784d-d70b-424e-ae43-a05e14c6ee19 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:35.879495 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9784d-d70b-424e-ae43-a05e14c6ee19 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:35.879772 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d9784d-d70b-424e-ae43-a05e14c6ee19 None admin] PATH_INFO: `/v3/projects/6da23d78618243288286bfd8d4b7c9d9` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:35.880917 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d9784d-d70b-424e-ae43-a05e14c6ee19 None admin] RBAC: Authorizing `identity:get_project(project_id=6da23d78618243288286bfd8d4b7c9d9)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:35.882930 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d9784d-d70b-424e-ae43-a05e14c6ee19 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:35.885307 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 95/184] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Oct 23 19:52:35 2022] GET /identity/v3/projects/6da23d78618243288286bfd8d4b7c9d9 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:35.893206 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d9784d-d70b-424e-ae43-a05e14c6ee19 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:35.920785 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415d76c2-4023-41f7-b09d-8222e9bcfc82 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:35.961502 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7660afc4-ff52-4802-acff-d943ec89f219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:35.962115 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660afc4-ff52-4802-acff-d943ec89f219 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:35.962432 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660afc4-ff52-4802-acff-d943ec89f219 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:35.962698 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7660afc4-ff52-4802-acff-d943ec89f219 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:35.963436 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7660afc4-ff52-4802-acff-d943ec89f219 None admin] RBAC: Adding query filter params (role.id=2989d29f392d4d8c802354159891eb11, scope.project.id=6da23d78618243288286bfd8d4b7c9d9, user.id=e5b99caa28074137863d87b3756ea9cc) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:35.963740 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7660afc4-ff52-4802-acff-d943ec89f219 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:35.965183 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7660afc4-ff52-4802-acff-d943ec89f219 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:35.977525 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 90/185] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Oct 23 19:52:35 2022] GET /identity/v3/role_assignments?user.id=e5b99caa28074137863d87b3756ea9cc&scope.project.id=6da23d78618243288286bfd8d4b7c9d9&role.id=2989d29f392d4d8c802354159891eb11 => generated 608 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:35.980475 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:36.061333 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d9784d-d70b-424e-ae43-a05e14c6ee19 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:36.116729 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087aa34f-13eb-4394-838e-c55ca6841729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:36.117372 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087aa34f-13eb-4394-838e-c55ca6841729 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:36.117649 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087aa34f-13eb-4394-838e-c55ca6841729 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:36.117919 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087aa34f-13eb-4394-838e-c55ca6841729 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:36.118717 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087aa34f-13eb-4394-838e-c55ca6841729 None admin] RBAC: Adding query filter params (role.id=742dfaf63ef74207a2c9213411513925, scope.project.id=6da23d78618243288286bfd8d4b7c9d9, user.id=e5b99caa28074137863d87b3756ea9cc) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:36.119026 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087aa34f-13eb-4394-838e-c55ca6841729 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:36.120563 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087aa34f-13eb-4394-838e-c55ca6841729 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:36.131608 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 96/186] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Oct 23 19:52:35 2022] GET /identity/v3/role_assignments?user.id=e5b99caa28074137863d87b3756ea9cc&scope.project.id=6da23d78618243288286bfd8d4b7c9d9&role.id=742dfaf63ef74207a2c9213411513925 => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:36.883216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa200d9b-c36b-4a26-9d70-1611bda420b3 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:36.883760 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa200d9b-c36b-4a26-9d70-1611bda420b3 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:36.883760 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa200d9b-c36b-4a26-9d70-1611bda420b3 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:36.885105 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 91/187] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:36.890929 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9125d59-b58b-41f3-87c0-82fb48847904 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:36.891412 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9125d59-b58b-41f3-87c0-82fb48847904 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:36.891560 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9125d59-b58b-41f3-87c0-82fb48847904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:36.909219 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-f9125d59-b58b-41f3-87c0-82fb48847904 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:36.910474 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f9125d59-b58b-41f3-87c0-82fb48847904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:36.986303 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 97/188] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:37.071870 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1cf6a-97dd-4467-8adf-b00498f2252e None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:37.072199 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1cf6a-97dd-4467-8adf-b00498f2252e None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:37.072423 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a1cf6a-97dd-4467-8adf-b00498f2252e None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:37.073801 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 92/189] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:37.079534 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8477f9-7727-4b06-8701-de6205c31657 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:37.079825 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8477f9-7727-4b06-8701-de6205c31657 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:37.080101 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8477f9-7727-4b06-8701-de6205c31657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:37.098454 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-cd8477f9-7727-4b06-8701-de6205c31657 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:37.099637 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-cd8477f9-7727-4b06-8701-de6205c31657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:37.142703 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:37.168475 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 98/190] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:37.256774 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a1cf6a-97dd-4467-8adf-b00498f2252e None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:37.310800 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:37.366179 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a1cf6a-97dd-4467-8adf-b00498f2252e None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:37.400429 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f40bd6-6154-4ed0-a136-90016d85d072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:37.400994 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f40bd6-6154-4ed0-a136-90016d85d072 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:37.401275 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f40bd6-6154-4ed0-a136-90016d85d072 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:37.401549 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f40bd6-6154-4ed0-a136-90016d85d072 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:37.412431 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f40bd6-6154-4ed0-a136-90016d85d072 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:37.415512 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b64b20-72e8-408a-917c-e9e201d06dad None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:37.415799 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b64b20-72e8-408a-917c-e9e201d06dad None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:37.416073 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b64b20-72e8-408a-917c-e9e201d06dad None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:37.417552 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 99/191] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 19:52:37.417668 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-d8f40bd6-6154-4ed0-a136-90016d85d072 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 19:52:37.417668 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:37.417668 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:37.417668 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:37.417668 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:37.417668 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:37.417668 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:37.417668 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:37.417668 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:37.417668 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:37.417668 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:37.417668 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:37.417668 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:37.417668 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:37.417668 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:37.417668 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:37.418760 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:37.418760 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:37.418760 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:37.418760 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:37.418760 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:37.418760 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:37.418760 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:37.418760 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:37.418760 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:37.418760 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:37.418760 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:37.418760 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:37.418760 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:37.418760 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:37.418760 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:37.418760 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:37.419582 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:37.419582 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:37.419582 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:37.419582 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:37.419582 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:37.419582 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:37.419582 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:37.419582 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:37.419582 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:37.419582 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:37.419582 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 19:52:37.419582 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:52:37.419582 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 93/192] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:52:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:52:37.422207 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8dc4a-539b-4066-985d-3c8259a93f49 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:37.422584 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8dc4a-539b-4066-985d-3c8259a93f49 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:37.422923 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f8dc4a-539b-4066-985d-3c8259a93f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:37.426091 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f40bd6-6154-4ed0-a136-90016d85d072 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:37.445978 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-d9f8dc4a-539b-4066-985d-3c8259a93f49 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:37.447244 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d9f8dc4a-539b-4066-985d-3c8259a93f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:37.512928 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 100/193] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:37.513588 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8dc4a-539b-4066-985d-3c8259a93f49 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:37.529437 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f40bd6-6154-4ed0-a136-90016d85d072 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:37.538352 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:37.566019 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c8f47a-d19c-437b-ba32-49bdd5ab7563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:37.567008 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8f47a-d19c-437b-ba32-49bdd5ab7563 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:37.567294 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8f47a-d19c-437b-ba32-49bdd5ab7563 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:37.567836 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c8f47a-d19c-437b-ba32-49bdd5ab7563 None admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:37.568506 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c8f47a-d19c-437b-ba32-49bdd5ab7563 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:37.568944 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c8f47a-d19c-437b-ba32-49bdd5ab7563 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:37.570677 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c8f47a-d19c-437b-ba32-49bdd5ab7563 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:37.581710 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 94/194] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Oct 23 19:52:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:37.586530 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c8f47a-d19c-437b-ba32-49bdd5ab7563 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:37.622012 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f8dc4a-539b-4066-985d-3c8259a93f49 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:37.629280 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:37.658542 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0fef12-578c-4769-b84d-c872f5e13d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:37.659105 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fef12-578c-4769-b84d-c872f5e13d88 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:37.659379 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fef12-578c-4769-b84d-c872f5e13d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:37.659651 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0fef12-578c-4769-b84d-c872f5e13d88 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:37.660735 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0fef12-578c-4769-b84d-c872f5e13d88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:37.662190 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0fef12-578c-4769-b84d-c872f5e13d88 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:37.663896 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 101/195] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:52:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:37.669168 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0fef12-578c-4769-b84d-c872f5e13d88 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:37.694003 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c8f47a-d19c-437b-ba32-49bdd5ab7563 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:37.728924 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c8bef-af95-4734-9e6b-26d9e93d89a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:37.729557 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c8bef-af95-4734-9e6b-26d9e93d89a4 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:37.729836 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c8bef-af95-4734-9e6b-26d9e93d89a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:37.730110 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93c8bef-af95-4734-9e6b-26d9e93d89a4 None admin] PATH_INFO: `/v3/users/2b9cee4f884b441b836435ab8c8c63e3` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:37.744832 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93c8bef-af95-4734-9e6b-26d9e93d89a4 None admin] RBAC: Authorizing `identity:get_user(user_id=2b9cee4f884b441b836435ab8c8c63e3)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:37.746322 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93c8bef-af95-4734-9e6b-26d9e93d89a4 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:37.752329 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 95/196] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:52:37 2022] GET /identity/v3/users/2b9cee4f884b441b836435ab8c8c63e3 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:37.753290 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c8bef-af95-4734-9e6b-26d9e93d89a4 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:37.779080 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0fef12-578c-4769-b84d-c872f5e13d88 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:37.782896 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:37.815646 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d415d9-37c5-404c-90bb-3eee1e179239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:37.816252 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d415d9-37c5-404c-90bb-3eee1e179239 None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:37.816532 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d415d9-37c5-404c-90bb-3eee1e179239 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:37.816803 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d415d9-37c5-404c-90bb-3eee1e179239 None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:37.817451 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d415d9-37c5-404c-90bb-3eee1e179239 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:37.818929 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d415d9-37c5-404c-90bb-3eee1e179239 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:37.839783 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 102/197] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Oct 23 19:52:37 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 19:52:37.840693 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d415d9-37c5-404c-90bb-3eee1e179239 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:37.862064 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93c8bef-af95-4734-9e6b-26d9e93d89a4 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:37.875945 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:37.902885 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e93229-3774-4d66-ac36-cfcef62f11a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:37.903656 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e93229-3774-4d66-ac36-cfcef62f11a1 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:37.904026 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e93229-3774-4d66-ac36-cfcef62f11a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:37.904372 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e93229-3774-4d66-ac36-cfcef62f11a1 None admin] PATH_INFO: `/v3/projects/6da23d78618243288286bfd8d4b7c9d9` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:37.905628 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e93229-3774-4d66-ac36-cfcef62f11a1 None admin] RBAC: Authorizing `identity:get_project(project_id=6da23d78618243288286bfd8d4b7c9d9)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:37.907748 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e93229-3774-4d66-ac36-cfcef62f11a1 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:37.909702 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 96/198] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Oct 23 19:52:37 2022] GET /identity/v3/projects/6da23d78618243288286bfd8d4b7c9d9 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:37.915285 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e93229-3774-4d66-ac36-cfcef62f11a1 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:37.957593 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d415d9-37c5-404c-90bb-3eee1e179239 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:37.995528 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07248327-b82e-43a7-bb32-9034eecbeaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:37.996180 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07248327-b82e-43a7-bb32-9034eecbeaa5 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:37.996464 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07248327-b82e-43a7-bb32-9034eecbeaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:37.996739 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07248327-b82e-43a7-bb32-9034eecbeaa5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:38.011322 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07248327-b82e-43a7-bb32-9034eecbeaa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:38.014442 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-07248327-b82e-43a7-bb32-9034eecbeaa5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 19:52:38.014442 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:38.014442 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:38.014442 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:38.014442 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:38.014442 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:38.014442 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:38.014442 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:38.014442 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:38.014442 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:38.014442 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:38.014442 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:38.014442 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:38.014442 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:38.014442 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:38.014442 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:38.015566 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:38.015566 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:38.015566 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:38.015566 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:38.015566 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:38.015566 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:38.015566 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:38.015566 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:38.015566 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:38.015566 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:38.015566 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:38.015566 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:38.015566 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:38.015566 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:38.015566 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:38.015566 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:38.016517 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:38.016517 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:38.016517 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:38.016517 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:38.016517 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:38.016517 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:38.016517 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:38.016517 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:38.016517 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:38.016517 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:38.016517 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 19:52:38.016517 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:52:38.016517 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 103/199] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:52:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:52:38.021334 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07248327-b82e-43a7-bb32-9034eecbeaa5 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:38.028592 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:38.035861 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e93229-3774-4d66-ac36-cfcef62f11a1 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:38.079918 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef9bf29-2b97-4fe1-909f-2045af6a1d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:38.080590 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9bf29-2b97-4fe1-909f-2045af6a1d72 None admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:38.080872 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9bf29-2b97-4fe1-909f-2045af6a1d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:38.081149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef9bf29-2b97-4fe1-909f-2045af6a1d72 None admin] PATH_INFO: `/v3/projects/6da23d78618243288286bfd8d4b7c9d9/users/2b9cee4f884b441b836435ab8c8c63e3/roles/54768ecdabbd443ca182db572b13b3ca` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:38.083640 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef9bf29-2b97-4fe1-909f-2045af6a1d72 None admin] RBAC: Authorizing `identity:create_grant(project_id=6da23d78618243288286bfd8d4b7c9d9, user_id=2b9cee4f884b441b836435ab8c8c63e3, role_id=54768ecdabbd443ca182db572b13b3ca)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:38.084638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef9bf29-2b97-4fe1-909f-2045af6a1d72 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:38.100284 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 97/200] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Oct 23 19:52:37 2022] PUT /identity/v3/projects/6da23d78618243288286bfd8d4b7c9d9/users/2b9cee4f884b441b836435ab8c8c63e3/roles/54768ecdabbd443ca182db572b13b3ca => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 19:52:38.139156 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:38.150378 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07248327-b82e-43a7-bb32-9034eecbeaa5 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:38.192792 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79450ef9-fd5a-40ce-857f-ee8fbc1a2cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:38.193453 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79450ef9-fd5a-40ce-857f-ee8fbc1a2cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:38.193744 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79450ef9-fd5a-40ce-857f-ee8fbc1a2cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:38.194022 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79450ef9-fd5a-40ce-857f-ee8fbc1a2cd2 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:38.194862 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79450ef9-fd5a-40ce-857f-ee8fbc1a2cd2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:38.195176 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79450ef9-fd5a-40ce-857f-ee8fbc1a2cd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:38.196682 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79450ef9-fd5a-40ce-857f-ee8fbc1a2cd2 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:38.208079 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 104/201] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Oct 23 19:52:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:38.214679 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef9bf29-2b97-4fe1-909f-2045af6a1d72 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:38.334273 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef9bf29-2b97-4fe1-909f-2045af6a1d72 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:38.370846 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f6776-23d5-45e7-9fe3-6e95fa6091b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:38.371488 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f6776-23d5-45e7-9fe3-6e95fa6091b0 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:38.371771 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f6776-23d5-45e7-9fe3-6e95fa6091b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:38.372051 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869f6776-23d5-45e7-9fe3-6e95fa6091b0 None admin] PATH_INFO: `/v3/users/cc2328fef2a347dc855f2823bcca687b` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:38.373203 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869f6776-23d5-45e7-9fe3-6e95fa6091b0 None admin] RBAC: Authorizing `identity:get_user(user_id=cc2328fef2a347dc855f2823bcca687b)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:38.374868 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869f6776-23d5-45e7-9fe3-6e95fa6091b0 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:38.376606 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 98/202] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:52:38 2022] GET /identity/v3/users/cc2328fef2a347dc855f2823bcca687b => generated 297 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:38.382549 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79450ef9-fd5a-40ce-857f-ee8fbc1a2cd2 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:38.490659 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79450ef9-fd5a-40ce-857f-ee8fbc1a2cd2 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:38.526787 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02742a55-2ff7-457c-8f53-d27eb05643e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:38.527759 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02742a55-2ff7-457c-8f53-d27eb05643e7 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:38.528237 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02742a55-2ff7-457c-8f53-d27eb05643e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:38.528725 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02742a55-2ff7-457c-8f53-d27eb05643e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:38.553307 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:38.602307 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02742a55-2ff7-457c-8f53-d27eb05643e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:38.603771 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02742a55-2ff7-457c-8f53-d27eb05643e7 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:38.709712 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 105/203] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Oct 23 19:52:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 19:52:38.711082 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f6776-23d5-45e7-9fe3-6e95fa6091b0 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:38.741432 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:38.821422 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869f6776-23d5-45e7-9fe3-6e95fa6091b0 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:38.857240 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e95ee44-697a-4913-b0d9-24c05f3c651a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:38.857872 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95ee44-697a-4913-b0d9-24c05f3c651a None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:38.858147 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95ee44-697a-4913-b0d9-24c05f3c651a None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:38.858448 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e95ee44-697a-4913-b0d9-24c05f3c651a None admin] PATH_INFO: `/v3/projects/6da23d78618243288286bfd8d4b7c9d9` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:38.859816 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e95ee44-697a-4913-b0d9-24c05f3c651a None admin] RBAC: Authorizing `identity:get_project(project_id=6da23d78618243288286bfd8d4b7c9d9)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:38.861258 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e95ee44-697a-4913-b0d9-24c05f3c651a None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:38.862948 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 99/204] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Oct 23 19:52:38 2022] GET /identity/v3/projects/6da23d78618243288286bfd8d4b7c9d9 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:38.868790 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02742a55-2ff7-457c-8f53-d27eb05643e7 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:38.924722 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:38.979114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02742a55-2ff7-457c-8f53-d27eb05643e7 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:39.017524 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e450e2-4534-47dd-b0b3-bca78d375696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:39.018067 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e450e2-4534-47dd-b0b3-bca78d375696 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:39.018378 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e450e2-4534-47dd-b0b3-bca78d375696 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:39.018656 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e450e2-4534-47dd-b0b3-bca78d375696 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:39.021182 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e450e2-4534-47dd-b0b3-bca78d375696 None admin] RBAC: Adding query filter params (role.id=69610ceae19c4a2b963433e2a9f5bb26, scope.project.id=6da23d78618243288286bfd8d4b7c9d9, user.id=cc2328fef2a347dc855f2823bcca687b) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:39.021182 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e450e2-4534-47dd-b0b3-bca78d375696 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:39.021182 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e450e2-4534-47dd-b0b3-bca78d375696 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:39.027275 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 106/205] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Oct 23 19:52:38 2022] GET /identity/v3/role_assignments?user.id=cc2328fef2a347dc855f2823bcca687b&scope.project.id=6da23d78618243288286bfd8d4b7c9d9&role.id=69610ceae19c4a2b963433e2a9f5bb26 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:40.162790 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088ed64-f53a-4fce-aafb-b11ced103155 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:40.163310 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088ed64-f53a-4fce-aafb-b11ced103155 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:40.163310 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e088ed64-f53a-4fce-aafb-b11ced103155 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:40.164745 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 100/206] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:40.171023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34815830-57ba-4e2d-8afc-19145d512a65 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:40.171526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34815830-57ba-4e2d-8afc-19145d512a65 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:40.171596 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34815830-57ba-4e2d-8afc-19145d512a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:40.192432 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-34815830-57ba-4e2d-8afc-19145d512a65 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:40.193651 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-34815830-57ba-4e2d-8afc-19145d512a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:40.268653 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 107/207] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:40.285038 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3d5ff-4749-48de-8662-46355efc5205 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:40.285262 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3d5ff-4749-48de-8662-46355efc5205 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:40.285529 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee3d5ff-4749-48de-8662-46355efc5205 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:40.287052 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 101/208] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:40.292041 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5e7ba-f284-457c-a7a1-ba323b63dbf3 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:40.292337 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5e7ba-f284-457c-a7a1-ba323b63dbf3 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:40.292616 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d5e7ba-f284-457c-a7a1-ba323b63dbf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:40.311032 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-70d5e7ba-f284-457c-a7a1-ba323b63dbf3 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:40.312215 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-70d5e7ba-f284-457c-a7a1-ba323b63dbf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:40.323740 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33406637-50dd-4f3f-8bbd-fe7833292415 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:40.324027 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33406637-50dd-4f3f-8bbd-fe7833292415 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:40.324299 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33406637-50dd-4f3f-8bbd-fe7833292415 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:40.325761 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 102/209] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 19:52:40.330442 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7ad0b-73e0-4b53-aeff-a610a783b79e None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:40.330723 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7ad0b-73e0-4b53-aeff-a610a783b79e None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:40.330994 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b7ad0b-73e0-4b53-aeff-a610a783b79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:40.349362 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-38b7ad0b-73e0-4b53-aeff-a610a783b79e None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:40.350570 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-38b7ad0b-73e0-4b53-aeff-a610a783b79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:40.358603 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:40.384222 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 108/210] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:40.416681 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 103/211] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 19:52:40.539756 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d5e7ba-f284-457c-a7a1-ba323b63dbf3 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:40.637952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d5e7ba-f284-457c-a7a1-ba323b63dbf3 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:40.651002 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b7ad0b-73e0-4b53-aeff-a610a783b79e None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:40.673880 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d96899-98ec-49cc-b62d-830cae7a20fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:40.675015 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:40.675958 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d96899-98ec-49cc-b62d-830cae7a20fe None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:40.676523 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d96899-98ec-49cc-b62d-830cae7a20fe None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:40.677084 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d96899-98ec-49cc-b62d-830cae7a20fe None admin] PATH_INFO: `/v3/roles/member` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:40.689396 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d96899-98ec-49cc-b62d-830cae7a20fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:40.694357 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-26d96899-98ec-49cc-b62d-830cae7a20fe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 19:52:40.694357 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:40.694357 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:40.694357 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:40.694357 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:40.694357 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:40.694357 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:40.694357 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:40.694357 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:40.694357 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:40.694357 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:40.694357 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:40.694357 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:40.694357 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:40.694357 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:40.694357 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:40.695197 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:40.695197 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:40.695197 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:40.695197 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:40.695197 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:40.695197 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:40.695197 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:40.695197 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:40.695197 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:40.695197 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:40.695197 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:40.695197 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:40.695197 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:40.695197 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:40.695197 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:40.695197 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:40.696028 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:40.696028 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:40.696028 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:40.696028 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:40.696028 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:40.696028 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:40.696028 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:40.696028 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:40.696028 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:40.696028 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:40.696028 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 19:52:40.696028 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:52:40.696028 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 109/212] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:52:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:52:40.696663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d96899-98ec-49cc-b62d-830cae7a20fe None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:40.730921 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:40.756530 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b7ad0b-73e0-4b53-aeff-a610a783b79e None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:40.789199 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13af29d-1209-4b5f-9e1c-087fa59ba1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:40.789871 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13af29d-1209-4b5f-9e1c-087fa59ba1bd None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:40.790097 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13af29d-1209-4b5f-9e1c-087fa59ba1bd None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:40.790395 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13af29d-1209-4b5f-9e1c-087fa59ba1bd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:40.791475 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13af29d-1209-4b5f-9e1c-087fa59ba1bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:40.792911 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13af29d-1209-4b5f-9e1c-087fa59ba1bd None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:40.794513 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 104/213] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:52:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:40.794513 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13af29d-1209-4b5f-9e1c-087fa59ba1bd None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:40.798453 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d96899-98ec-49cc-b62d-830cae7a20fe None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:40.832394 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c202ec-a2b9-4135-8852-623d6a583983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:40.833021 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c202ec-a2b9-4135-8852-623d6a583983 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:40.833300 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c202ec-a2b9-4135-8852-623d6a583983 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:40.833574 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c202ec-a2b9-4135-8852-623d6a583983 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:40.843639 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c202ec-a2b9-4135-8852-623d6a583983 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:40.848949 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-32c202ec-a2b9-4135-8852-623d6a583983 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 19:52:40.848949 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:40.848949 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:40.848949 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:40.848949 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:40.848949 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:40.848949 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:40.848949 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:40.848949 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:40.848949 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:40.848949 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:40.848949 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:40.848949 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:40.848949 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:40.848949 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:40.848949 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:40.849872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:40.849872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:40.849872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:40.849872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:40.849872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:40.849872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:40.849872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:40.849872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:40.849872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:40.849872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:40.849872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:40.849872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:40.849872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:40.849872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:40.849872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:40.849872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:40.850721 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:40.850721 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:40.850721 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:40.850721 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:40.850721 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:40.850721 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:40.850721 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:40.850721 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:40.850721 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:40.850721 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:40.850721 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 19:52:40.850721 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:52:40.850721 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 110/214] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:52:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:52:40.855150 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c202ec-a2b9-4135-8852-623d6a583983 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:40.883828 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:40.894011 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13af29d-1209-4b5f-9e1c-087fa59ba1bd None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:40.927524 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eff1b0-e4a2-462d-99b3-0a4e5fb2dd1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:40.928120 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eff1b0-e4a2-462d-99b3-0a4e5fb2dd1b None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:40.928398 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eff1b0-e4a2-462d-99b3-0a4e5fb2dd1b None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:40.928669 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eff1b0-e4a2-462d-99b3-0a4e5fb2dd1b None admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:40.929329 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eff1b0-e4a2-462d-99b3-0a4e5fb2dd1b None admin] RBAC: Adding query filter params (name=member) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:40.929631 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eff1b0-e4a2-462d-99b3-0a4e5fb2dd1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:40.931020 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94eff1b0-e4a2-462d-99b3-0a4e5fb2dd1b None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:40.939764 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 105/215] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Oct 23 19:52:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:40.940543 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eff1b0-e4a2-462d-99b3-0a4e5fb2dd1b None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:40.950760 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:40.957194 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c202ec-a2b9-4135-8852-623d6a583983 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:40.991040 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a97e731-27e6-4aa5-9963-55ea22833cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:40.991647 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97e731-27e6-4aa5-9963-55ea22833cd6 None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:40.991933 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97e731-27e6-4aa5-9963-55ea22833cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:40.992206 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a97e731-27e6-4aa5-9963-55ea22833cd6 None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:40.992851 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a97e731-27e6-4aa5-9963-55ea22833cd6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:40.994312 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a97e731-27e6-4aa5-9963-55ea22833cd6 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:41.018952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 111/216] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Oct 23 19:52:40 2022] POST /identity/v3/users => generated 293 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:52:41.023304 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a97e731-27e6-4aa5-9963-55ea22833cd6 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:41.044794 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eff1b0-e4a2-462d-99b3-0a4e5fb2dd1b None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:41.085254 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1f5937-7317-4039-9b77-3cbf72d9f9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:41.085927 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f5937-7317-4039-9b77-3cbf72d9f9ff None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:41.086202 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f5937-7317-4039-9b77-3cbf72d9f9ff None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:41.086504 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f5937-7317-4039-9b77-3cbf72d9f9ff None admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:41.087189 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1f5937-7317-4039-9b77-3cbf72d9f9ff None admin] RBAC: Adding query filter params (name=reader) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:41.087497 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1f5937-7317-4039-9b77-3cbf72d9f9ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:41.088929 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1f5937-7317-4039-9b77-3cbf72d9f9ff None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:41.102961 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 106/217] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Oct 23 19:52:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:41.104255 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1f5937-7317-4039-9b77-3cbf72d9f9ff None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:41.123492 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:41.135354 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a97e731-27e6-4aa5-9963-55ea22833cd6 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:41.180769 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c9267-eb44-4ebb-9701-d52391f4ad88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:41.181470 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c9267-eb44-4ebb-9701-d52391f4ad88 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:41.181689 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c9267-eb44-4ebb-9701-d52391f4ad88 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:41.181967 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c9267-eb44-4ebb-9701-d52391f4ad88 None admin] PATH_INFO: `/v3/users/9c66a6fda1da4745ba4724fed05e4fca` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:41.201763 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762c9267-eb44-4ebb-9701-d52391f4ad88 None admin] RBAC: Authorizing `identity:get_user(user_id=9c66a6fda1da4745ba4724fed05e4fca)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:41.203898 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762c9267-eb44-4ebb-9701-d52391f4ad88 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:41.207906 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 112/218] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:52:41 2022] GET /identity/v3/users/9c66a6fda1da4745ba4724fed05e4fca => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:41.213441 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c9267-eb44-4ebb-9701-d52391f4ad88 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:41.221743 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1f5937-7317-4039-9b77-3cbf72d9f9ff None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:41.226836 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:41.265182 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918b70ac-4b57-4c54-bbfd-e80ba7f66645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:41.266422 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b70ac-4b57-4c54-bbfd-e80ba7f66645 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:41.266708 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b70ac-4b57-4c54-bbfd-e80ba7f66645 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:41.266983 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b70ac-4b57-4c54-bbfd-e80ba7f66645 None admin] PATH_INFO: `/v3/users/2b9cee4f884b441b836435ab8c8c63e3` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:41.268132 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918b70ac-4b57-4c54-bbfd-e80ba7f66645 None admin] RBAC: Authorizing `identity:get_user(user_id=2b9cee4f884b441b836435ab8c8c63e3)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:41.269591 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918b70ac-4b57-4c54-bbfd-e80ba7f66645 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:41.271292 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 107/219] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:52:41 2022] GET /identity/v3/users/2b9cee4f884b441b836435ab8c8c63e3 => generated 297 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:41.277591 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918b70ac-4b57-4c54-bbfd-e80ba7f66645 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:41.336868 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c9267-eb44-4ebb-9701-d52391f4ad88 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:41.382885 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:41.387282 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9852ce-1749-4060-844a-bc6594855086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:41.387907 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9852ce-1749-4060-844a-bc6594855086 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:41.388176 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9852ce-1749-4060-844a-bc6594855086 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:41.388449 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9852ce-1749-4060-844a-bc6594855086 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:41.389192 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9852ce-1749-4060-844a-bc6594855086 None admin] RBAC: Adding query filter params (role.id=69610ceae19c4a2b963433e2a9f5bb26, user.id=9c66a6fda1da4745ba4724fed05e4fca, scope.system=all) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:41.389500 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9852ce-1749-4060-844a-bc6594855086 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:41.395008 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9852ce-1749-4060-844a-bc6594855086 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:41.397268 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 113/220] 10.222.0.13 () {56 vars in 1308 bytes} [Sun Oct 23 19:52:41 2022] GET /identity/v3/role_assignments?user.id=9c66a6fda1da4745ba4724fed05e4fca&scope.system=all&role.id=69610ceae19c4a2b963433e2a9f5bb26 => generated 226 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:41.409338 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918b70ac-4b57-4c54-bbfd-e80ba7f66645 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:41.449004 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9759f-a7db-4784-a3b7-b4e1bf46bc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:41.449630 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9759f-a7db-4784-a3b7-b4e1bf46bc37 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:41.449912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9759f-a7db-4784-a3b7-b4e1bf46bc37 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:41.450190 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed9759f-a7db-4784-a3b7-b4e1bf46bc37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:41.527699 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed9759f-a7db-4784-a3b7-b4e1bf46bc37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:41.529185 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed9759f-a7db-4784-a3b7-b4e1bf46bc37 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:41.619572 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:41.639674 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 108/221] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Oct 23 19:52:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 19:52:41.645517 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9852ce-1749-4060-844a-bc6594855086 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:41.729205 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:41.755209 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9852ce-1749-4060-844a-bc6594855086 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:41.789778 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5ad4c6-3ade-4a12-9984-add0a0d2be88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:41.790440 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ad4c6-3ade-4a12-9984-add0a0d2be88 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:41.790723 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ad4c6-3ade-4a12-9984-add0a0d2be88 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:41.790992 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5ad4c6-3ade-4a12-9984-add0a0d2be88 None admin] PATH_INFO: `/v3/projects/6da23d78618243288286bfd8d4b7c9d9` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:41.792103 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5ad4c6-3ade-4a12-9984-add0a0d2be88 None admin] RBAC: Authorizing `identity:get_project(project_id=6da23d78618243288286bfd8d4b7c9d9)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:41.793578 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5ad4c6-3ade-4a12-9984-add0a0d2be88 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:41.795560 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 114/222] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Oct 23 19:52:41 2022] GET /identity/v3/projects/6da23d78618243288286bfd8d4b7c9d9 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:41.801436 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9759f-a7db-4784-a3b7-b4e1bf46bc37 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:41.915278 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed9759f-a7db-4784-a3b7-b4e1bf46bc37 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:41.952196 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb8cc0-ebe1-4f3a-9070-43e02573fb39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:41.952809 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb8cc0-ebe1-4f3a-9070-43e02573fb39 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:41.953122 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb8cc0-ebe1-4f3a-9070-43e02573fb39 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:41.953397 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb8cc0-ebe1-4f3a-9070-43e02573fb39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:41.954130 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb8cc0-ebe1-4f3a-9070-43e02573fb39 None admin] RBAC: Adding query filter params (role.id=54768ecdabbd443ca182db572b13b3ca, scope.project.id=6da23d78618243288286bfd8d4b7c9d9, user.id=2b9cee4f884b441b836435ab8c8c63e3) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:41.954465 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb8cc0-ebe1-4f3a-9070-43e02573fb39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:41.955902 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb8cc0-ebe1-4f3a-9070-43e02573fb39 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:41.962232 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 109/223] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Oct 23 19:52:41 2022] GET /identity/v3/role_assignments?user.id=2b9cee4f884b441b836435ab8c8c63e3&scope.project.id=6da23d78618243288286bfd8d4b7c9d9&role.id=54768ecdabbd443ca182db572b13b3ca => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:43.354455 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4da03-d99b-4b41-b487-d65dbc83b8ae None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:43.354998 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4da03-d99b-4b41-b487-d65dbc83b8ae None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:43.354998 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b4da03-d99b-4b41-b487-d65dbc83b8ae None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:43.356496 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 115/224] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:43.361962 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99956ad-fe48-4512-a227-5a46c382477b None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:43.362406 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99956ad-fe48-4512-a227-5a46c382477b None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:43.362578 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99956ad-fe48-4512-a227-5a46c382477b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:43.383716 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-c99956ad-fe48-4512-a227-5a46c382477b None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:43.384908 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-c99956ad-fe48-4512-a227-5a46c382477b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:43.398619 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c20b9-818c-4126-99c0-b0a6f0040079 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:43.398794 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c20b9-818c-4126-99c0-b0a6f0040079 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:43.399054 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552c20b9-818c-4126-99c0-b0a6f0040079 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:43.400553 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 116/225] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:43.405168 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c4f65-8121-4048-9832-a81bcd7e9a8c None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:43.405456 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c4f65-8121-4048-9832-a81bcd7e9a8c None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:43.405736 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c4f65-8121-4048-9832-a81bcd7e9a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:43.427101 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-256c4f65-8121-4048-9832-a81bcd7e9a8c None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:43.428296 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-256c4f65-8121-4048-9832-a81bcd7e9a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:43.433845 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:43.461856 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 110/226] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:43.499380 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 117/227] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 19:52:43.629036 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63085cf-650a-491f-bd23-96fbb24d469d None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:43.629343 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63085cf-650a-491f-bd23-96fbb24d469d None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:43.629516 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63085cf-650a-491f-bd23-96fbb24d469d None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:43.630949 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 111/228] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:43.636116 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ddefc-d5dd-4339-8a23-d5ecf09dedff None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:43.636473 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ddefc-d5dd-4339-8a23-d5ecf09dedff None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:43.636818 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ddefc-d5dd-4339-8a23-d5ecf09dedff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:43.655108 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-5e9ddefc-d5dd-4339-8a23-d5ecf09dedff None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:43.656468 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5e9ddefc-d5dd-4339-8a23-d5ecf09dedff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:43.722743 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 118/229] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:43.728300 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63085cf-650a-491f-bd23-96fbb24d469d None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:43.788845 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ddefc-d5dd-4339-8a23-d5ecf09dedff None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:43.792398 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:43.832960 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63085cf-650a-491f-bd23-96fbb24d469d None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:43.867624 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a08d6-c523-48fc-827c-3b5a0d104415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:43.868261 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a08d6-c523-48fc-827c-3b5a0d104415 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:43.868545 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a08d6-c523-48fc-827c-3b5a0d104415 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:43.868819 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a08d6-c523-48fc-827c-3b5a0d104415 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:43.881507 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a08d6-c523-48fc-827c-3b5a0d104415 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:43.886603 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-c98a08d6-c523-48fc-827c-3b5a0d104415 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 19:52:43.886603 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:43.886603 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:43.886603 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:43.886603 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:43.886603 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:43.886603 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:43.886603 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:43.886603 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:43.886603 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:43.886603 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:43.886603 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:43.886603 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:43.886603 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:43.886603 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:43.886603 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:43.887509 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:43.887509 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:43.887509 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:43.887509 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:43.887509 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:43.887509 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:43.887509 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:43.887509 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:43.887509 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:43.887509 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:43.887509 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:43.887509 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:43.887509 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:43.887509 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:43.887509 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:43.887509 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:43.888309 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:43.888309 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:43.888309 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:43.888309 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:43.888309 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:43.888309 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:43.888309 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:43.888309 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:43.888309 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:43.888309 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:43.888309 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 19:52:43.888309 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:52:43.893539 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 112/230] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:52:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:52:43.894455 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a08d6-c523-48fc-827c-3b5a0d104415 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:43.902196 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ddefc-d5dd-4339-8a23-d5ecf09dedff None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:43.936999 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:43.939218 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c39bdb-64f7-401b-83cd-a7e9c0d44e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:43.939933 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c39bdb-64f7-401b-83cd-a7e9c0d44e56 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:43.940230 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c39bdb-64f7-401b-83cd-a7e9c0d44e56 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:43.940489 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c39bdb-64f7-401b-83cd-a7e9c0d44e56 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:43.941608 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c39bdb-64f7-401b-83cd-a7e9c0d44e56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:43.943077 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c39bdb-64f7-401b-83cd-a7e9c0d44e56 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:43.944817 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 119/231] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:52:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:43.950363 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c39bdb-64f7-401b-83cd-a7e9c0d44e56 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:44.005543 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a08d6-c523-48fc-827c-3b5a0d104415 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:44.024641 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:44.040669 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8170819c-b59f-432f-8a46-ffb370a81615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:44.041272 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170819c-b59f-432f-8a46-ffb370a81615 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:44.041547 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170819c-b59f-432f-8a46-ffb370a81615 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:44.041817 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8170819c-b59f-432f-8a46-ffb370a81615 None admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:44.042501 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8170819c-b59f-432f-8a46-ffb370a81615 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:44.042800 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8170819c-b59f-432f-8a46-ffb370a81615 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:44.044173 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8170819c-b59f-432f-8a46-ffb370a81615 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:44.052818 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 113/232] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Oct 23 19:52:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:44.052818 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8170819c-b59f-432f-8a46-ffb370a81615 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:44.057779 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c39bdb-64f7-401b-83cd-a7e9c0d44e56 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:44.092342 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07734470-3d37-472c-8048-1c35909015ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:44.092957 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07734470-3d37-472c-8048-1c35909015ba None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:44.093238 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07734470-3d37-472c-8048-1c35909015ba None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:44.093536 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07734470-3d37-472c-8048-1c35909015ba None admin] PATH_INFO: `/v3/groups` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:44.094176 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07734470-3d37-472c-8048-1c35909015ba None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:44.095644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07734470-3d37-472c-8048-1c35909015ba None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:44.110578 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 120/233] 10.222.0.13 () {60 vars in 1134 bytes} [Sun Oct 23 19:52:43 2022] POST /identity/v3/groups => generated 229 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:52:44.126507 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07734470-3d37-472c-8048-1c35909015ba None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:44.160136 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:44.165894 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8170819c-b59f-432f-8a46-ffb370a81615 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:44.204366 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11918a22-78bf-4691-9402-39135e95d075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:44.205033 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11918a22-78bf-4691-9402-39135e95d075 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:44.205317 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11918a22-78bf-4691-9402-39135e95d075 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:44.205593 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11918a22-78bf-4691-9402-39135e95d075 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:44.216945 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11918a22-78bf-4691-9402-39135e95d075 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:44.223035 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-11918a22-78bf-4691-9402-39135e95d075 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 19:52:44.223035 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:44.223035 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:44.223035 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:44.223035 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:44.223035 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:44.223035 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:44.223035 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:44.223035 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:44.223035 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:44.223035 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:44.223035 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:44.223035 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:44.223035 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:44.223035 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:44.223035 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:44.224004 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:44.224004 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:44.224004 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:44.224004 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:44.224004 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:44.224004 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:44.224004 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:44.224004 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:44.224004 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:44.224004 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:44.224004 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:44.224004 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:44.224004 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:44.224004 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:44.224004 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:44.224004 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:44.224781 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:44.224781 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:44.224781 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:44.224781 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:44.224781 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:44.224781 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:44.224781 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:44.224781 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:44.224781 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:44.224781 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:44.224781 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 19:52:44.224781 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:52:44.224781 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 114/234] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:52:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:52:44.230107 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11918a22-78bf-4691-9402-39135e95d075 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:44.233138 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07734470-3d37-472c-8048-1c35909015ba None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:44.266946 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:44.275222 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11787840-3a21-4de1-893c-2ead7968d683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:44.276205 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11787840-3a21-4de1-893c-2ead7968d683 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:44.276723 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11787840-3a21-4de1-893c-2ead7968d683 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:44.277263 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11787840-3a21-4de1-893c-2ead7968d683 None admin] PATH_INFO: `/v3/users/dfe31d26b5524db88d5c560c0c44423e` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:44.292222 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11787840-3a21-4de1-893c-2ead7968d683 None admin] RBAC: Authorizing `identity:get_user(user_id=dfe31d26b5524db88d5c560c0c44423e)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:44.293690 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11787840-3a21-4de1-893c-2ead7968d683 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:44.295423 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 121/235] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:52:44 2022] GET /identity/v3/users/dfe31d26b5524db88d5c560c0c44423e => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:44.301727 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11787840-3a21-4de1-893c-2ead7968d683 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:44.352184 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11918a22-78bf-4691-9402-39135e95d075 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:44.392520 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c370336c-8fbc-460f-9e35-5e827f5f24b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:44.393945 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370336c-8fbc-460f-9e35-5e827f5f24b2 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:44.394462 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370336c-8fbc-460f-9e35-5e827f5f24b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:44.394963 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c370336c-8fbc-460f-9e35-5e827f5f24b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:44.395881 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c370336c-8fbc-460f-9e35-5e827f5f24b2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:44.396257 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c370336c-8fbc-460f-9e35-5e827f5f24b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:44.397640 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c370336c-8fbc-460f-9e35-5e827f5f24b2 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:44.410360 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 115/236] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Oct 23 19:52:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:44.411772 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c370336c-8fbc-460f-9e35-5e827f5f24b2 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:44.426320 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11787840-3a21-4de1-893c-2ead7968d683 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:44.436318 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:44.465226 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b990f61-710d-499d-b213-b97583cea89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:44.465866 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b990f61-710d-499d-b213-b97583cea89a None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:44.466117 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b990f61-710d-499d-b213-b97583cea89a None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:44.466404 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b990f61-710d-499d-b213-b97583cea89a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:44.467126 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b990f61-710d-499d-b213-b97583cea89a None admin] RBAC: Adding query filter params (role.id=54768ecdabbd443ca182db572b13b3ca, user.id=dfe31d26b5524db88d5c560c0c44423e, scope.system=all) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:44.467430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b990f61-710d-499d-b213-b97583cea89a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:44.468913 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b990f61-710d-499d-b213-b97583cea89a None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:44.473909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 122/237] 10.222.0.13 () {56 vars in 1308 bytes} [Sun Oct 23 19:52:44 2022] GET /identity/v3/role_assignments?user.id=dfe31d26b5524db88d5c560c0c44423e&scope.system=all&role.id=54768ecdabbd443ca182db572b13b3ca => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:44.524996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c370336c-8fbc-460f-9e35-5e827f5f24b2 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:44.561981 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f47394-0f41-44ec-983f-46473ffd9e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:44.563482 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f47394-0f41-44ec-983f-46473ffd9e00 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:44.563766 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f47394-0f41-44ec-983f-46473ffd9e00 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:44.564039 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f47394-0f41-44ec-983f-46473ffd9e00 None admin] PATH_INFO: `/v3/users/9c66a6fda1da4745ba4724fed05e4fca` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:44.565237 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f47394-0f41-44ec-983f-46473ffd9e00 None admin] RBAC: Authorizing `identity:get_user(user_id=9c66a6fda1da4745ba4724fed05e4fca)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:44.566757 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f47394-0f41-44ec-983f-46473ffd9e00 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:44.568713 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 116/238] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:52:44 2022] GET /identity/v3/users/9c66a6fda1da4745ba4724fed05e4fca => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:44.574219 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b990f61-710d-499d-b213-b97583cea89a None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:44.632621 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:44.686747 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b990f61-710d-499d-b213-b97583cea89a None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:44.722293 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ce1fa1-5b2a-4554-a6da-cb6d343069fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:44.723538 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce1fa1-5b2a-4554-a6da-cb6d343069fe None admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:44.723872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce1fa1-5b2a-4554-a6da-cb6d343069fe None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:44.724225 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ce1fa1-5b2a-4554-a6da-cb6d343069fe None admin] PATH_INFO: `/v3/system/users/9c66a6fda1da4745ba4724fed05e4fca/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:44.725740 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ce1fa1-5b2a-4554-a6da-cb6d343069fe None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=9c66a6fda1da4745ba4724fed05e4fca, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:44.727135 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ce1fa1-5b2a-4554-a6da-cb6d343069fe None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:44.736486 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 123/239] 10.222.0.13 () {58 vars in 1334 bytes} [Sun Oct 23 19:52:44 2022] PUT /identity/v3/system/users/9c66a6fda1da4745ba4724fed05e4fca/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 19:52:46.294154 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9339c9-4923-40df-9b2b-2895aa422a41 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:46.294669 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9339c9-4923-40df-9b2b-2895aa422a41 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:46.294669 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9339c9-4923-40df-9b2b-2895aa422a41 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:46.296038 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 117/240] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:46.301151 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9cfa9a-3b72-476a-81b1-8108100a0c5e None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:46.301536 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9cfa9a-3b72-476a-81b1-8108100a0c5e None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:46.301706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9cfa9a-3b72-476a-81b1-8108100a0c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:46.320469 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-4a9cfa9a-3b72-476a-81b1-8108100a0c5e None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:46.321642 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4a9cfa9a-3b72-476a-81b1-8108100a0c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:46.388220 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 124/241] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:46.602521 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb566a4-bd09-47af-b19f-67d57efc64dd None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:46.602871 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb566a4-bd09-47af-b19f-67d57efc64dd None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:46.603077 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb566a4-bd09-47af-b19f-67d57efc64dd None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:46.604567 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 118/242] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:46.610106 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7500718-be39-4ff0-b569-69290a707ac8 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:46.610423 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7500718-be39-4ff0-b569-69290a707ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:46.610693 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7500718-be39-4ff0-b569-69290a707ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:46.629078 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-d7500718-be39-4ff0-b569-69290a707ac8 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:46.630268 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d7500718-be39-4ff0-b569-69290a707ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:46.657753 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb566a4-bd09-47af-b19f-67d57efc64dd None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:46.672056 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:46.701407 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 125/243] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:46.754175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:46.768344 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb566a4-bd09-47af-b19f-67d57efc64dd None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:46.806213 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341bd373-1ddd-42b8-9e86-c29bb91abd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:46.806885 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bd373-1ddd-42b8-9e86-c29bb91abd75 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:46.807158 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bd373-1ddd-42b8-9e86-c29bb91abd75 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:46.807427 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bd373-1ddd-42b8-9e86-c29bb91abd75 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:46.808525 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341bd373-1ddd-42b8-9e86-c29bb91abd75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:46.810015 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341bd373-1ddd-42b8-9e86-c29bb91abd75 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:46.816933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 119/244] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:52:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:46.818267 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7500718-be39-4ff0-b569-69290a707ac8 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:46.894554 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015836f7-fa73-467b-9720-07ec8b28de6f None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:46.894756 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015836f7-fa73-467b-9720-07ec8b28de6f None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:46.895043 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015836f7-fa73-467b-9720-07ec8b28de6f None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:46.896520 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 120/245] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 19:52:46.901140 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f21bfa-79df-42c7-976c-dc96e0f5131a None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:46.901422 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f21bfa-79df-42c7-976c-dc96e0f5131a None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:46.901698 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f21bfa-79df-42c7-976c-dc96e0f5131a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:46.918308 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7500718-be39-4ff0-b569-69290a707ac8 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:46.919919 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-23f21bfa-79df-42c7-976c-dc96e0f5131a None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:46.921089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-23f21bfa-79df-42c7-976c-dc96e0f5131a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:46.954929 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac35d9-c5c5-4a11-a011-32cf4b534c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:46.955531 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac35d9-c5c5-4a11-a011-32cf4b534c90 None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:46.955804 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac35d9-c5c5-4a11-a011-32cf4b534c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:46.956089 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ac35d9-c5c5-4a11-a011-32cf4b534c90 None admin] PATH_INFO: `/v3/groups` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:46.956695 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ac35d9-c5c5-4a11-a011-32cf4b534c90 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:46.958117 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ac35d9-c5c5-4a11-a011-32cf4b534c90 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:46.974101 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 126/246] 10.222.0.13 () {60 vars in 1134 bytes} [Sun Oct 23 19:52:46 2022] POST /identity/v3/groups => generated 226 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 19:52:46.974871 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac35d9-c5c5-4a11-a011-32cf4b534c90 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:46.994560 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 121/247] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 19:52:47.026341 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:47.100445 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ac35d9-c5c5-4a11-a011-32cf4b534c90 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:47.137088 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b526c1-fa33-496e-aef7-7a6b5cd99f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:47.137725 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b526c1-fa33-496e-aef7-7a6b5cd99f77 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:47.138003 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b526c1-fa33-496e-aef7-7a6b5cd99f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:47.138293 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b526c1-fa33-496e-aef7-7a6b5cd99f77 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:47.150184 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b526c1-fa33-496e-aef7-7a6b5cd99f77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:47.154998 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-63b526c1-fa33-496e-aef7-7a6b5cd99f77 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 19:52:47.154998 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:47.154998 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:47.154998 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:47.154998 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:47.154998 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:47.154998 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:47.154998 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:47.154998 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:47.154998 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:47.154998 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:47.154998 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:47.154998 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:47.154998 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:47.154998 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:47.154998 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:47.155873 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:47.155873 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:47.155873 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:47.155873 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:47.155873 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:47.155873 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:47.155873 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:47.155873 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:47.155873 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:47.155873 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:47.155873 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:47.155873 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:47.155873 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:47.155873 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:47.155873 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:47.155873 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:47.157270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:47.157270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:47.157270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:47.157270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:47.157270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:47.157270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:47.157270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:47.157270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:47.157270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:47.157270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:47.157270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 19:52:47.157270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:52:47.157270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 127/248] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:52:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:52:47.162402 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f21bfa-79df-42c7-976c-dc96e0f5131a None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:47.253309 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:47.266348 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b526c1-fa33-496e-aef7-7a6b5cd99f77 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:47.272843 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f21bfa-79df-42c7-976c-dc96e0f5131a None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:47.316669 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de7f38-8acf-486c-8e3c-190ddd79b054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:47.317318 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de7f38-8acf-486c-8e3c-190ddd79b054 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:47.317607 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de7f38-8acf-486c-8e3c-190ddd79b054 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:47.317883 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6de7f38-8acf-486c-8e3c-190ddd79b054 None admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:47.318612 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6de7f38-8acf-486c-8e3c-190ddd79b054 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:47.318918 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6de7f38-8acf-486c-8e3c-190ddd79b054 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:47.320311 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6de7f38-8acf-486c-8e3c-190ddd79b054 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:47.331283 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 122/249] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Oct 23 19:52:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:47.338711 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de7f38-8acf-486c-8e3c-190ddd79b054 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:47.382579 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:47.390097 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b526c1-fa33-496e-aef7-7a6b5cd99f77 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:47.441202 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0f470-790d-43d8-8c17-e31becf87c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:47.441757 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0f470-790d-43d8-8c17-e31becf87c91 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:47.442033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0f470-790d-43d8-8c17-e31becf87c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:47.442380 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc0f470-790d-43d8-8c17-e31becf87c91 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:47.460238 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc0f470-790d-43d8-8c17-e31becf87c91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:47.467203 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-cfc0f470-790d-43d8-8c17-e31becf87c91 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 19:52:47.467203 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:47.467203 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:47.467203 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:47.467203 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:47.467203 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:47.467203 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:47.467203 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:47.467203 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:47.467203 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:47.467203 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:47.467203 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:47.467203 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:47.467203 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:47.467203 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:47.467203 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:47.468087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:47.468087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:47.468087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:47.468087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:47.468087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:47.468087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:47.468087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:47.468087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:47.468087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:47.468087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:47.468087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:47.468087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:47.468087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:47.468087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:47.468087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:47.468087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:47.469457 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:47.469457 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:47.469457 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:47.469457 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:47.469457 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:47.469457 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:47.469457 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:47.469457 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:47.469457 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:47.469457 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:47.469457 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 19:52:47.469457 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:52:47.469457 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 128/250] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:52:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:52:47.471342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6de7f38-8acf-486c-8e3c-190ddd79b054 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:47.477266 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0f470-790d-43d8-8c17-e31becf87c91 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:47.515858 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aa17c5-7bd6-4e53-a0a0-7faaf81f6d61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:47.516881 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa17c5-7bd6-4e53-a0a0-7faaf81f6d61 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:47.517646 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa17c5-7bd6-4e53-a0a0-7faaf81f6d61 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:47.518114 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa17c5-7bd6-4e53-a0a0-7faaf81f6d61 None admin] PATH_INFO: `/v3/users/dfe31d26b5524db88d5c560c0c44423e` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:47.520923 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aa17c5-7bd6-4e53-a0a0-7faaf81f6d61 None admin] RBAC: Authorizing `identity:get_user(user_id=dfe31d26b5524db88d5c560c0c44423e)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:47.522627 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5aa17c5-7bd6-4e53-a0a0-7faaf81f6d61 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:47.525455 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 123/251] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:52:47 2022] GET /identity/v3/users/dfe31d26b5524db88d5c560c0c44423e => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:47.532287 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aa17c5-7bd6-4e53-a0a0-7faaf81f6d61 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:47.555860 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:47.608767 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc0f470-790d-43d8-8c17-e31becf87c91 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:47.649110 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5aa17c5-7bd6-4e53-a0a0-7faaf81f6d61 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:47.654344 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360bc149-c6d4-48ad-bb14-1db9aa336fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:47.655138 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360bc149-c6d4-48ad-bb14-1db9aa336fe4 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:47.655560 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360bc149-c6d4-48ad-bb14-1db9aa336fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:47.656032 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360bc149-c6d4-48ad-bb14-1db9aa336fe4 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:47.657343 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360bc149-c6d4-48ad-bb14-1db9aa336fe4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:47.657801 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360bc149-c6d4-48ad-bb14-1db9aa336fe4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:47.659902 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360bc149-c6d4-48ad-bb14-1db9aa336fe4 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:47.670811 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 129/252] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Oct 23 19:52:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:47.676386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360bc149-c6d4-48ad-bb14-1db9aa336fe4 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:47.696681 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73affee6-abca-479b-ae53-399cf087c7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:47.697350 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73affee6-abca-479b-ae53-399cf087c7b1 None admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:47.697717 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73affee6-abca-479b-ae53-399cf087c7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:47.698064 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73affee6-abca-479b-ae53-399cf087c7b1 None admin] PATH_INFO: `/v3/system/users/dfe31d26b5524db88d5c560c0c44423e/roles/54768ecdabbd443ca182db572b13b3ca` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:47.699545 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73affee6-abca-479b-ae53-399cf087c7b1 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=dfe31d26b5524db88d5c560c0c44423e, role_id=54768ecdabbd443ca182db572b13b3ca)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:47.701054 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73affee6-abca-479b-ae53-399cf087c7b1 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:47.712689 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 124/253] 10.222.0.13 () {58 vars in 1334 bytes} [Sun Oct 23 19:52:47 2022] PUT /identity/v3/system/users/dfe31d26b5524db88d5c560c0c44423e/roles/54768ecdabbd443ca182db572b13b3ca => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 19:52:47.714616 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:47.810738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360bc149-c6d4-48ad-bb14-1db9aa336fe4 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:47.861641 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b0595-c6de-409e-9e16-c306de740e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:47.862610 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b0595-c6de-409e-9e16-c306de740e35 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:47.863141 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b0595-c6de-409e-9e16-c306de740e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:47.863570 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b0595-c6de-409e-9e16-c306de740e35 None admin] PATH_INFO: `/v3/users/9c66a6fda1da4745ba4724fed05e4fca` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:47.866064 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b0595-c6de-409e-9e16-c306de740e35 None admin] RBAC: Authorizing `identity:get_user(user_id=9c66a6fda1da4745ba4724fed05e4fca)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:47.867832 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b0595-c6de-409e-9e16-c306de740e35 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:47.870101 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 130/254] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:52:47 2022] GET /identity/v3/users/9c66a6fda1da4745ba4724fed05e4fca => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:47.876243 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73affee6-abca-479b-ae53-399cf087c7b1 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:47.967992 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:47.991713 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73affee6-abca-479b-ae53-399cf087c7b1 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:48.040124 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89250deb-8ed9-430e-8cfc-8bfa53b409a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:48.041019 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89250deb-8ed9-430e-8cfc-8bfa53b409a2 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:48.041439 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89250deb-8ed9-430e-8cfc-8bfa53b409a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:48.041858 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89250deb-8ed9-430e-8cfc-8bfa53b409a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:48.042798 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89250deb-8ed9-430e-8cfc-8bfa53b409a2 None admin] RBAC: Adding query filter params (role.id=69610ceae19c4a2b963433e2a9f5bb26, user.id=9c66a6fda1da4745ba4724fed05e4fca, scope.system=all) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:48.043245 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89250deb-8ed9-430e-8cfc-8bfa53b409a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:48.044839 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89250deb-8ed9-430e-8cfc-8bfa53b409a2 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:48.058143 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 125/255] 10.222.0.13 () {56 vars in 1308 bytes} [Sun Oct 23 19:52:47 2022] GET /identity/v3/role_assignments?user.id=9c66a6fda1da4745ba4724fed05e4fca&scope.system=all&role.id=69610ceae19c4a2b963433e2a9f5bb26 => generated 510 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:49.533523 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603c752-4722-4ed8-a7e3-979d958b5fc3 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:49.534057 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603c752-4722-4ed8-a7e3-979d958b5fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:49.534057 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7603c752-4722-4ed8-a7e3-979d958b5fc3 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:49.535449 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 131/256] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:49.540999 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831481e2-8c77-471f-8a59-4361e52001f7 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:49.541810 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831481e2-8c77-471f-8a59-4361e52001f7 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:49.542323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831481e2-8c77-471f-8a59-4361e52001f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:49.567666 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-831481e2-8c77-471f-8a59-4361e52001f7 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:49.569234 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-831481e2-8c77-471f-8a59-4361e52001f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:49.632490 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6419ec-a070-45ff-be86-55f1335664e1 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:49.632679 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6419ec-a070-45ff-be86-55f1335664e1 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:49.632928 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6419ec-a070-45ff-be86-55f1335664e1 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:49.634308 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 132/257] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 19:52:49.641054 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b223c7d-15be-4779-bae7-35364c6e35d4 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:49.641336 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b223c7d-15be-4779-bae7-35364c6e35d4 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:49.641612 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b223c7d-15be-4779-bae7-35364c6e35d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:49.643853 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 126/258] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:49 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 23 19:52:49.683870 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-6b223c7d-15be-4779-bae7-35364c6e35d4 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:49.685104 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-6b223c7d-15be-4779-bae7-35364c6e35d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:49.700564 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 133/259] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:49 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 23 19:52:50.170645 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831481e2-8c77-471f-8a59-4361e52001f7 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:50.180918 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831481e2-8c77-471f-8a59-4361e52001f7 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:50.188752 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55af3ac-8900-4289-b207-56fe3235c772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:50.189672 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55af3ac-8900-4289-b207-56fe3235c772 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:50.190156 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55af3ac-8900-4289-b207-56fe3235c772 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:50.190660 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55af3ac-8900-4289-b207-56fe3235c772 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:50.201936 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55af3ac-8900-4289-b207-56fe3235c772 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:50.207134 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b223c7d-15be-4779-bae7-35364c6e35d4 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:50.211416 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-f55af3ac-8900-4289-b207-56fe3235c772 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 19:52:50.211416 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:50.211416 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:50.211416 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:50.211416 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:50.211416 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:50.211416 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:50.211416 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:50.211416 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:50.211416 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:50.211416 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:50.211416 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:50.211416 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:50.211416 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:50.211416 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:50.211416 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:50.212294 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:50.212294 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:50.212294 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:50.212294 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:50.212294 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:50.212294 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:50.212294 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:50.212294 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:50.212294 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:50.212294 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:50.212294 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:50.212294 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:50.212294 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:50.212294 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:50.212294 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:50.212294 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:50.213079 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:50.213079 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:50.213079 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:50.213079 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:50.213079 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:50.213079 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:50.213079 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:50.213079 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:50.213079 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:50.213079 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:50.213079 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 19:52:50.213079 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:52:50.215981 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 127/260] 10.222.0.13 () {56 vars in 1135 bytes} [Sun Oct 23 19:52:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 46 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:52:50.221304 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b223c7d-15be-4779-bae7-35364c6e35d4 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:50.222223 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55af3ac-8900-4289-b207-56fe3235c772 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:50.227292 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8766d34-9e5f-4c83-b201-6bb99a7c3667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:50.227994 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8766d34-9e5f-4c83-b201-6bb99a7c3667 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:50.228276 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8766d34-9e5f-4c83-b201-6bb99a7c3667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:50.228553 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8766d34-9e5f-4c83-b201-6bb99a7c3667 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:50.234591 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55af3ac-8900-4289-b207-56fe3235c772 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:50.242398 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebbf5c3-0a98-435c-806f-06f85c940f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:50.243278 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebbf5c3-0a98-435c-806f-06f85c940f8f admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:50.243760 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebbf5c3-0a98-435c-806f-06f85c940f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:50.244232 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebbf5c3-0a98-435c-806f-06f85c940f8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:50.245125 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebbf5c3-0a98-435c-806f-06f85c940f8f admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:50.245628 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebbf5c3-0a98-435c-806f-06f85c940f8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:50.247638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: WARNING py.warnings [None req-4ebbf5c3-0a98-435c-806f-06f85c940f8f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 23 19:52:50.247638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: warnings.warn(msg) Oct 23 19:52:50.247638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: Oct 23 19:52:50.248910 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebbf5c3-0a98-435c-806f-06f85c940f8f admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:50.250882 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8766d34-9e5f-4c83-b201-6bb99a7c3667 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:50.256376 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-e8766d34-9e5f-4c83-b201-6bb99a7c3667 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 19:52:50.256376 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:50.256376 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:50.256376 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:50.256376 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:50.256376 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:50.256376 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:50.256376 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:50.256376 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:50.256376 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:50.256376 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:50.256376 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:50.256376 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:50.256376 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:50.256376 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:50.256376 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:50.257247 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:50.257247 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:50.257247 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:50.257247 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:50.257247 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:50.257247 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:50.257247 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:50.257247 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:50.257247 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:50.257247 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:50.257247 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:50.257247 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:50.257247 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:50.257247 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:50.257247 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:50.257247 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:50.262673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:50.262673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:50.262673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:50.262673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:50.262673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:50.262673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:50.262673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:50.262673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:50.262673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:50.262673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:50.262673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 19:52:50.262673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:52:50.262673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 134/261] 10.222.0.13 () {56 vars in 1120 bytes} [Sun Oct 23 19:52:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 51 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:52:50.263384 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8766d34-9e5f-4c83-b201-6bb99a7c3667 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:50.266633 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 128/262] 10.222.0.13 () {56 vars in 1132 bytes} [Sun Oct 23 19:52:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:50.272204 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebbf5c3-0a98-435c-806f-06f85c940f8f admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:50.280079 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8766d34-9e5f-4c83-b201-6bb99a7c3667 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:50.282644 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebbf5c3-0a98-435c-806f-06f85c940f8f admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:50.292859 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d7f33-71d3-4ae2-a962-77e2dcb4995f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:50.293795 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d7f33-71d3-4ae2-a962-77e2dcb4995f admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:50.294328 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d7f33-71d3-4ae2-a962-77e2dcb4995f admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:50.294802 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4d7f33-71d3-4ae2-a962-77e2dcb4995f admin admin] PATH_INFO: `/v3/groups/4358694d61434fb99fe9bb5be26fb3a4` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:50.294921 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4363fbf-0bb2-4ab1-ac55-74cb8182a76b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:50.295581 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4363fbf-0bb2-4ab1-ac55-74cb8182a76b admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:50.295865 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4363fbf-0bb2-4ab1-ac55-74cb8182a76b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:50.296137 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4363fbf-0bb2-4ab1-ac55-74cb8182a76b admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:50.297363 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4363fbf-0bb2-4ab1-ac55-74cb8182a76b admin admin] RBAC: Adding query filter params (name=member) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:50.297670 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4363fbf-0bb2-4ab1-ac55-74cb8182a76b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:50.299636 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: WARNING py.warnings [None req-d4363fbf-0bb2-4ab1-ac55-74cb8182a76b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 23 19:52:50.299636 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: warnings.warn(msg) Oct 23 19:52:50.299636 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: Oct 23 19:52:50.300568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4363fbf-0bb2-4ab1-ac55-74cb8182a76b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:50.304613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d7f33-71d3-4ae2-a962-77e2dcb4995f admin admin] RBAC: Authorizing `identity:get_group(group_id=4358694d61434fb99fe9bb5be26fb3a4)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:50.305925 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: WARNING py.warnings [None req-6d4d7f33-71d3-4ae2-a962-77e2dcb4995f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 19:52:50.305925 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: warnings.warn(msg) Oct 23 19:52:50.305925 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: Oct 23 19:52:50.307550 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4d7f33-71d3-4ae2-a962-77e2dcb4995f admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:50.309502 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 129/263] 10.222.0.13 () {56 vars in 1201 bytes} [Sun Oct 23 19:52:50 2022] GET /identity/v3/groups/4358694d61434fb99fe9bb5be26fb3a4 => generated 226 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:50.314976 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 135/264] 10.222.0.13 () {56 vars in 1122 bytes} [Sun Oct 23 19:52:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:50.316237 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d7f33-71d3-4ae2-a962-77e2dcb4995f admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:50.320427 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4363fbf-0bb2-4ab1-ac55-74cb8182a76b admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:50.329803 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4d7f33-71d3-4ae2-a962-77e2dcb4995f admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:50.336659 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4363fbf-0bb2-4ab1-ac55-74cb8182a76b admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:50.340379 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e11519-8f68-471a-99bc-a4a0a240b4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:50.341264 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e11519-8f68-471a-99bc-a4a0a240b4ba admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:50.341868 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e11519-8f68-471a-99bc-a4a0a240b4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:50.342369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e11519-8f68-471a-99bc-a4a0a240b4ba admin admin] PATH_INFO: `/v3/projects/0a195ad6e19d43ba991afe5458887d8c` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:50.343817 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe9a32-1b09-40c3-892a-ac331d0fa11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:50.344120 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e11519-8f68-471a-99bc-a4a0a240b4ba admin admin] RBAC: Authorizing `identity:get_project(project_id=0a195ad6e19d43ba991afe5458887d8c)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:50.344456 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe9a32-1b09-40c3-892a-ac331d0fa11b admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:50.344737 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe9a32-1b09-40c3-892a-ac331d0fa11b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:50.345007 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afe9a32-1b09-40c3-892a-ac331d0fa11b admin admin] PATH_INFO: `/v3/groups/4358694d61434fb99fe9bb5be26fb3a4` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:50.347095 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e11519-8f68-471a-99bc-a4a0a240b4ba admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:50.348894 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe9a32-1b09-40c3-892a-ac331d0fa11b admin admin] RBAC: Authorizing `identity:get_group(group_id=4358694d61434fb99fe9bb5be26fb3a4)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:50.349189 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 130/265] 10.222.0.13 () {56 vars in 1207 bytes} [Sun Oct 23 19:52:50 2022] GET /identity/v3/projects/0a195ad6e19d43ba991afe5458887d8c => generated 298 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:50.350171 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: WARNING py.warnings [None req-4afe9a32-1b09-40c3-892a-ac331d0fa11b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 19:52:50.350171 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: warnings.warn(msg) Oct 23 19:52:50.350171 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: Oct 23 19:52:50.351204 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afe9a32-1b09-40c3-892a-ac331d0fa11b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:50.352871 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 136/266] 10.222.0.13 () {56 vars in 1201 bytes} [Sun Oct 23 19:52:50 2022] GET /identity/v3/groups/4358694d61434fb99fe9bb5be26fb3a4 => generated 226 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:50.354928 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e11519-8f68-471a-99bc-a4a0a240b4ba admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:50.358076 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe9a32-1b09-40c3-892a-ac331d0fa11b admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:50.366213 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e11519-8f68-471a-99bc-a4a0a240b4ba admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:50.377156 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afe9a32-1b09-40c3-892a-ac331d0fa11b admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:50.378889 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a592975e-5b8e-440c-8453-d5c8aac92531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:50.379968 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592975e-5b8e-440c-8453-d5c8aac92531 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:50.380456 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592975e-5b8e-440c-8453-d5c8aac92531 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:50.380947 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a592975e-5b8e-440c-8453-d5c8aac92531 admin admin] PATH_INFO: `/v3/projects/0a195ad6e19d43ba991afe5458887d8c/groups/4358694d61434fb99fe9bb5be26fb3a4/roles/2989d29f392d4d8c802354159891eb11` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:50.383341 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a592975e-5b8e-440c-8453-d5c8aac92531 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a195ad6e19d43ba991afe5458887d8c, group_id=4358694d61434fb99fe9bb5be26fb3a4, role_id=2989d29f392d4d8c802354159891eb11)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:50.384561 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: WARNING py.warnings [None req-a592975e-5b8e-440c-8453-d5c8aac92531 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 19:52:50.384561 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: warnings.warn(msg) Oct 23 19:52:50.384561 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: Oct 23 19:52:50.387396 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a592975e-5b8e-440c-8453-d5c8aac92531 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:50.389127 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a199dd-b749-404e-bc08-9faad4d74c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:50.389767 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a199dd-b749-404e-bc08-9faad4d74c67 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:50.390104 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a199dd-b749-404e-bc08-9faad4d74c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:50.390440 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a199dd-b749-404e-bc08-9faad4d74c67 admin admin] PATH_INFO: `/v3/projects/0a195ad6e19d43ba991afe5458887d8c` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:50.393540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a199dd-b749-404e-bc08-9faad4d74c67 admin admin] RBAC: Authorizing `identity:get_project(project_id=0a195ad6e19d43ba991afe5458887d8c)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:50.396657 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a199dd-b749-404e-bc08-9faad4d74c67 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:50.403617 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 137/267] 10.222.0.13 () {56 vars in 1207 bytes} [Sun Oct 23 19:52:50 2022] GET /identity/v3/projects/0a195ad6e19d43ba991afe5458887d8c => generated 298 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:50.405879 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 131/268] 10.222.0.13 () {58 vars in 1463 bytes} [Sun Oct 23 19:52:50 2022] PUT /identity/v3/projects/0a195ad6e19d43ba991afe5458887d8c/groups/4358694d61434fb99fe9bb5be26fb3a4/roles/2989d29f392d4d8c802354159891eb11 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 19:52:50.410050 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a199dd-b749-404e-bc08-9faad4d74c67 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:50.424469 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a199dd-b749-404e-bc08-9faad4d74c67 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:50.495258 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:50.503265 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed48048d-3dae-4e67-9e88-d158781bd968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:50.504056 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48048d-3dae-4e67-9e88-d158781bd968 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:50.504343 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48048d-3dae-4e67-9e88-d158781bd968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:50.504665 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed48048d-3dae-4e67-9e88-d158781bd968 admin admin] PATH_INFO: `/v3/projects/0a195ad6e19d43ba991afe5458887d8c/groups/4358694d61434fb99fe9bb5be26fb3a4/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:50.507826 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed48048d-3dae-4e67-9e88-d158781bd968 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a195ad6e19d43ba991afe5458887d8c, group_id=4358694d61434fb99fe9bb5be26fb3a4, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:50.508847 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: WARNING py.warnings [None req-ed48048d-3dae-4e67-9e88-d158781bd968 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 19:52:50.508847 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: warnings.warn(msg) Oct 23 19:52:50.508847 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: Oct 23 19:52:50.510905 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed48048d-3dae-4e67-9e88-d158781bd968 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:50.530625 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 138/269] 10.222.0.13 () {58 vars in 1463 bytes} [Sun Oct 23 19:52:50 2022] PUT /identity/v3/projects/0a195ad6e19d43ba991afe5458887d8c/groups/4358694d61434fb99fe9bb5be26fb3a4/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 19:52:50.882825 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75bf95b-f55d-489a-a13b-623393146d6c None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:50.883829 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75bf95b-f55d-489a-a13b-623393146d6c None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:50.884340 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75bf95b-f55d-489a-a13b-623393146d6c None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:50.886111 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 132/270] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:50.891701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92784a1a-22b3-47c9-9288-94555b326483 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:50.892149 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92784a1a-22b3-47c9-9288-94555b326483 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:50.892334 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92784a1a-22b3-47c9-9288-94555b326483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:50.918309 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-92784a1a-22b3-47c9-9288-94555b326483 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:50.919613 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-92784a1a-22b3-47c9-9288-94555b326483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:50.986537 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 139/271] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:50 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 23 19:52:51.343312 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad03739-73a7-4418-ae2e-9f6765d7e2c7 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:51.344041 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad03739-73a7-4418-ae2e-9f6765d7e2c7 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:51.344556 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad03739-73a7-4418-ae2e-9f6765d7e2c7 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:51.346458 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 133/272] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:51.351585 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15942380-4bfa-444c-a0dd-eb762276e9ae None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:51.352114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15942380-4bfa-444c-a0dd-eb762276e9ae None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:51.352591 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15942380-4bfa-444c-a0dd-eb762276e9ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:51.379299 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-15942380-4bfa-444c-a0dd-eb762276e9ae None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:51.380838 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-15942380-4bfa-444c-a0dd-eb762276e9ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:51.467722 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 140/273] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:51.546628 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad03739-73a7-4418-ae2e-9f6765d7e2c7 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:51.564491 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad03739-73a7-4418-ae2e-9f6765d7e2c7 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:51.569416 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25cada-2ace-4882-a528-500a6102ec51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:51.570320 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25cada-2ace-4882-a528-500a6102ec51 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:51.570805 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25cada-2ace-4882-a528-500a6102ec51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:51.571276 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba25cada-2ace-4882-a528-500a6102ec51 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:51.583189 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba25cada-2ace-4882-a528-500a6102ec51 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:51.588763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-ba25cada-2ace-4882-a528-500a6102ec51 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 19:52:51.588763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:51.588763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:51.588763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:51.588763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:51.588763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:51.588763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:51.588763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:51.588763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:51.588763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:51.588763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:51.588763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:51.588763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:51.588763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:51.588763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:51.588763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:51.589613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:51.589613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:51.589613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:51.589613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:51.589613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:51.589613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:51.589613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:51.589613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:51.589613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:51.589613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:51.589613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:51.589613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:51.589613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:51.589613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:51.589613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:51.589613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:51.594867 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:51.594867 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:51.594867 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:51.594867 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:51.594867 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:51.594867 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:51.594867 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:51.594867 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:51.594867 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:51.594867 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:51.594867 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 19:52:51.594867 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:52:51.596892 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 134/274] 10.222.0.13 () {56 vars in 1120 bytes} [Sun Oct 23 19:52:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 51 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:52:51.602610 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15942380-4bfa-444c-a0dd-eb762276e9ae None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:51.614134 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15942380-4bfa-444c-a0dd-eb762276e9ae None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:51.621905 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94562d0-b201-4e52-a384-5e4788e033b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:51.622764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94562d0-b201-4e52-a384-5e4788e033b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:51.623248 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94562d0-b201-4e52-a384-5e4788e033b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:51.623722 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94562d0-b201-4e52-a384-5e4788e033b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:51.624582 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94562d0-b201-4e52-a384-5e4788e033b6 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:51.625128 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94562d0-b201-4e52-a384-5e4788e033b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:51.627006 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94562d0-b201-4e52-a384-5e4788e033b6 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:51.639360 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 141/275] 10.222.0.13 () {56 vars in 1122 bytes} [Sun Oct 23 19:52:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:51.644917 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25cada-2ace-4882-a528-500a6102ec51 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:51.655274 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba25cada-2ace-4882-a528-500a6102ec51 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:51.666368 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28634453-9c36-45b3-ac21-2a2dd4c57106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:51.667277 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28634453-9c36-45b3-ac21-2a2dd4c57106 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:51.667782 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28634453-9c36-45b3-ac21-2a2dd4c57106 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:51.668257 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28634453-9c36-45b3-ac21-2a2dd4c57106 admin admin] PATH_INFO: `/v3/groups/4358694d61434fb99fe9bb5be26fb3a4` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:51.669619 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28634453-9c36-45b3-ac21-2a2dd4c57106 admin admin] RBAC: Authorizing `identity:get_group(group_id=4358694d61434fb99fe9bb5be26fb3a4)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:51.671891 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28634453-9c36-45b3-ac21-2a2dd4c57106 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:51.673711 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 135/276] 10.222.0.13 () {56 vars in 1201 bytes} [Sun Oct 23 19:52:51 2022] GET /identity/v3/groups/4358694d61434fb99fe9bb5be26fb3a4 => generated 226 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:51.679451 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94562d0-b201-4e52-a384-5e4788e033b6 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:51.691165 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94562d0-b201-4e52-a384-5e4788e033b6 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:51.702014 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112ba123-d57c-4902-8b79-23cf1198d22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:51.703046 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ba123-d57c-4902-8b79-23cf1198d22d admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:51.703531 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ba123-d57c-4902-8b79-23cf1198d22d admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:51.704002 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112ba123-d57c-4902-8b79-23cf1198d22d admin admin] PATH_INFO: `/v3/projects/6da23d78618243288286bfd8d4b7c9d9` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:51.705382 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112ba123-d57c-4902-8b79-23cf1198d22d admin admin] RBAC: Authorizing `identity:get_project(project_id=6da23d78618243288286bfd8d4b7c9d9)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:51.707707 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112ba123-d57c-4902-8b79-23cf1198d22d admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:51.709570 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 142/277] 10.222.0.13 () {56 vars in 1207 bytes} [Sun Oct 23 19:52:51 2022] GET /identity/v3/projects/6da23d78618243288286bfd8d4b7c9d9 => generated 302 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:51.715164 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28634453-9c36-45b3-ac21-2a2dd4c57106 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:51.730262 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28634453-9c36-45b3-ac21-2a2dd4c57106 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:51.735482 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014c8840-a78b-49ce-86b4-64ab7f039278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:51.736425 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c8840-a78b-49ce-86b4-64ab7f039278 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:51.736909 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c8840-a78b-49ce-86b4-64ab7f039278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:51.737415 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014c8840-a78b-49ce-86b4-64ab7f039278 admin admin] PATH_INFO: `/v3/projects/6da23d78618243288286bfd8d4b7c9d9/groups/4358694d61434fb99fe9bb5be26fb3a4/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:51.739561 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014c8840-a78b-49ce-86b4-64ab7f039278 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6da23d78618243288286bfd8d4b7c9d9, group_id=4358694d61434fb99fe9bb5be26fb3a4, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:51.742485 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014c8840-a78b-49ce-86b4-64ab7f039278 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:51.761141 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 136/278] 10.222.0.13 () {58 vars in 1463 bytes} [Sun Oct 23 19:52:51 2022] PUT /identity/v3/projects/6da23d78618243288286bfd8d4b7c9d9/groups/4358694d61434fb99fe9bb5be26fb3a4/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 19:52:51.894336 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7f820-9952-4f3f-977e-7f2c8737c938 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:51.895276 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7f820-9952-4f3f-977e-7f2c8737c938 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:51.895765 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c7f820-9952-4f3f-977e-7f2c8737c938 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:51.897359 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 143/279] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:51.904429 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acf1ee-d2e6-406c-8609-7978d26d1b14 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:51.905288 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acf1ee-d2e6-406c-8609-7978d26d1b14 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:51.905784 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83acf1ee-d2e6-406c-8609-7978d26d1b14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:51.929269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-83acf1ee-d2e6-406c-8609-7978d26d1b14 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:51.930854 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-83acf1ee-d2e6-406c-8609-7978d26d1b14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:51.942529 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:51.974642 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b966f4b-21e0-41a0-b16a-f81fb8e2d584 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:51.975279 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b966f4b-21e0-41a0-b16a-f81fb8e2d584 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:51.975759 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b966f4b-21e0-41a0-b16a-f81fb8e2d584 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:51.978182 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 144/280] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:51.995245 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6020c2-01d6-48b5-8f48-5214ac815126 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:51.996552 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6020c2-01d6-48b5-8f48-5214ac815126 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:51.997058 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6020c2-01d6-48b5-8f48-5214ac815126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:52.031207 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-3a6020c2-01d6-48b5-8f48-5214ac815126 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:52.038741 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3a6020c2-01d6-48b5-8f48-5214ac815126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:52.043917 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 137/281] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:51 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 23 19:52:52.049221 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 145/282] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:51 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 23 19:52:52.099779 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acf1ee-d2e6-406c-8609-7978d26d1b14 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:52.235550 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83acf1ee-d2e6-406c-8609-7978d26d1b14 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:52.282507 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc0f20-6e02-4702-a9d8-79870229e1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:52.283566 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc0f20-6e02-4702-a9d8-79870229e1c7 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:52.284051 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc0f20-6e02-4702-a9d8-79870229e1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:52.284528 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc0f20-6e02-4702-a9d8-79870229e1c7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:52.300591 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fc0f20-6e02-4702-a9d8-79870229e1c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:52.305841 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-f9fc0f20-6e02-4702-a9d8-79870229e1c7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 19:52:52.305841 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:52.305841 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:52.305841 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:52.305841 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:52.305841 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:52.305841 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:52.305841 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:52.305841 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:52.305841 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:52.305841 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:52.305841 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:52.305841 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:52.305841 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:52.305841 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:52.305841 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:52.306766 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:52.306766 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:52.306766 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:52.306766 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:52.306766 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:52.306766 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:52.306766 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:52.306766 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:52.306766 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:52.306766 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:52.306766 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:52.306766 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:52.306766 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:52.306766 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:52.306766 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:52.306766 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:52.307636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:52.307636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:52.307636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:52.307636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:52.307636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:52.307636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:52.307636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:52.307636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:52.307636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:52.307636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:52.307636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 19:52:52.307636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:52:52.310222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 138/283] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:52:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:52:52.317259 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6020c2-01d6-48b5-8f48-5214ac815126 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:52.393008 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc0f20-6e02-4702-a9d8-79870229e1c7 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:52.399475 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:52.412576 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc0f20-6e02-4702-a9d8-79870229e1c7 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:52.420792 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2dae91-adcf-4c59-8043-a67d2d3c435d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:52.421754 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2dae91-adcf-4c59-8043-a67d2d3c435d admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:52.422239 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2dae91-adcf-4c59-8043-a67d2d3c435d admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:52.422762 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2dae91-adcf-4c59-8043-a67d2d3c435d admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:52.445160 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2dae91-adcf-4c59-8043-a67d2d3c435d admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:52.451178 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-9b2dae91-adcf-4c59-8043-a67d2d3c435d admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:52:52.451178 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:52.451178 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:52.451178 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:52.451178 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:52.451178 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:52.451178 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:52.451178 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:52.451178 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:52.451178 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:52.451178 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:52.451178 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:52.451178 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:52.451178 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:52.451178 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:52.451178 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:52.452046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:52.452046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:52.452046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:52.452046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:52.452046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:52.452046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:52.452046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:52.452046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:52.452046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:52.452046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:52.452046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:52.452046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:52.452046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:52.452046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:52.452046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:52.452046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:52.453424 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:52.453424 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:52.453424 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:52.453424 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:52.453424 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:52.453424 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:52.453424 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:52.453424 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:52.453424 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:52.453424 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:52.453424 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:52:52.453424 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:52:52.455893 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 139/284] 10.222.0.13 () {56 vars in 1117 bytes} [Sun Oct 23 19:52:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 63 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:52:52.456920 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2dae91-adcf-4c59-8043-a67d2d3c435d admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:52.466391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:52.471469 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2dae91-adcf-4c59-8043-a67d2d3c435d admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:52.479403 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6020c2-01d6-48b5-8f48-5214ac815126 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:52.483586 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb2a21-e080-4dbf-a472-ef84cf2156e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:52.484493 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb2a21-e080-4dbf-a472-ef84cf2156e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:52.485116 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb2a21-e080-4dbf-a472-ef84cf2156e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:52.486321 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb2a21-e080-4dbf-a472-ef84cf2156e1 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:52.509229 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb2a21-e080-4dbf-a472-ef84cf2156e1 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:52.519834 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-c2fb2a21-e080-4dbf-a472-ef84cf2156e1 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 19:52:52.519834 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:52.519834 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:52.519834 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:52.519834 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:52.519834 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:52.519834 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:52.519834 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:52.519834 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:52.519834 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:52.519834 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:52.519834 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:52.519834 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:52.519834 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:52.519834 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:52.519834 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:52.520703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:52.520703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:52.520703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:52.520703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:52.520703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:52.520703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:52.520703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:52.520703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:52.520703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:52.520703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:52.520703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:52.520703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:52.520703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:52.520703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:52.520703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:52.520703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:52.521499 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:52.521499 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:52.521499 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:52.521499 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:52.521499 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:52.521499 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:52.521499 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:52.521499 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:52.521499 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:52.521499 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:52.521499 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 19:52:52.521499 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:52:52.524604 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 140/285] 10.222.0.13 () {56 vars in 1135 bytes} [Sun Oct 23 19:52:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 68 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:52:52.528554 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb2a21-e080-4dbf-a472-ef84cf2156e1 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:52.549802 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb2a21-e080-4dbf-a472-ef84cf2156e1 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:52.557432 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ebcad2-6e10-4f66-a477-595c84d6f971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:52.558271 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ebcad2-6e10-4f66-a477-595c84d6f971 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:52.558702 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ebcad2-6e10-4f66-a477-595c84d6f971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:52.559145 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ebcad2-6e10-4f66-a477-595c84d6f971 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:52.560036 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ebcad2-6e10-4f66-a477-595c84d6f971 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:52.560565 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ebcad2-6e10-4f66-a477-595c84d6f971 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:52.561890 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcdfa95-8aab-481c-8937-30c9fdb439e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:52.562615 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ebcad2-6e10-4f66-a477-595c84d6f971 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:52.562848 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcdfa95-8aab-481c-8937-30c9fdb439e4 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:52.563493 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcdfa95-8aab-481c-8937-30c9fdb439e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:52.563869 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcdfa95-8aab-481c-8937-30c9fdb439e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:52.564717 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcdfa95-8aab-481c-8937-30c9fdb439e4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:52.568405 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcdfa95-8aab-481c-8937-30c9fdb439e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:52.568405 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcdfa95-8aab-481c-8937-30c9fdb439e4 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:52.581561 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 146/286] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Oct 23 19:52:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:52.582431 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcdfa95-8aab-481c-8937-30c9fdb439e4 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:52.585109 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 141/287] 10.222.0.13 () {56 vars in 1120 bytes} [Sun Oct 23 19:52:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:52.590329 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ebcad2-6e10-4f66-a477-595c84d6f971 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:52.600894 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcdfa95-8aab-481c-8937-30c9fdb439e4 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:52.611299 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca8cc4c-b79d-4076-868c-b84a52235b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:52.612239 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8cc4c-b79d-4076-868c-b84a52235b84 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:52.612239 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8cc4c-b79d-4076-868c-b84a52235b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:52.612424 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca8cc4c-b79d-4076-868c-b84a52235b84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:52.613104 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca8cc4c-b79d-4076-868c-b84a52235b84 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:52.613411 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca8cc4c-b79d-4076-868c-b84a52235b84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:52.615075 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca8cc4c-b79d-4076-868c-b84a52235b84 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:52.629401 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 147/288] 10.222.0.13 () {56 vars in 1132 bytes} [Sun Oct 23 19:52:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:52.630825 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca8cc4c-b79d-4076-868c-b84a52235b84 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:52.649116 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca8cc4c-b79d-4076-868c-b84a52235b84 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:52.662576 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a94236-98d2-4fe6-bd9c-93be301baf66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:52.663563 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a94236-98d2-4fe6-bd9c-93be301baf66 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:52.664088 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a94236-98d2-4fe6-bd9c-93be301baf66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:52.664584 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a94236-98d2-4fe6-bd9c-93be301baf66 admin admin] PATH_INFO: `/v3/groups/a36eacffd2ff41acb073c4d45a457ab9` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:52.678469 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a94236-98d2-4fe6-bd9c-93be301baf66 admin admin] RBAC: Authorizing `identity:get_group(group_id=a36eacffd2ff41acb073c4d45a457ab9)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:52.681356 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a94236-98d2-4fe6-bd9c-93be301baf66 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:52.686524 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 148/289] 10.222.0.13 () {56 vars in 1201 bytes} [Sun Oct 23 19:52:52 2022] GET /identity/v3/groups/a36eacffd2ff41acb073c4d45a457ab9 => generated 229 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:52.690209 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a94236-98d2-4fe6-bd9c-93be301baf66 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:52.708890 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a94236-98d2-4fe6-bd9c-93be301baf66 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:52.720867 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcd462-1e71-433e-835f-166850e80619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:52.723253 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcd462-1e71-433e-835f-166850e80619 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:52.723766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcd462-1e71-433e-835f-166850e80619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:52.725086 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcd462-1e71-433e-835f-166850e80619 admin admin] PATH_INFO: `/v3/groups/4358694d61434fb99fe9bb5be26fb3a4` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:52.728572 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcd462-1e71-433e-835f-166850e80619 admin admin] RBAC: Authorizing `identity:get_group(group_id=4358694d61434fb99fe9bb5be26fb3a4)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:52.734747 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcd462-1e71-433e-835f-166850e80619 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:52.736806 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 149/290] 10.222.0.13 () {56 vars in 1201 bytes} [Sun Oct 23 19:52:52 2022] GET /identity/v3/groups/4358694d61434fb99fe9bb5be26fb3a4 => generated 226 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:52.737970 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcd462-1e71-433e-835f-166850e80619 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:52.750095 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcd462-1e71-433e-835f-166850e80619 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:52.758155 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1979a26-cbae-4817-b92a-26dbb8f4e805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:52.759800 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1979a26-cbae-4817-b92a-26dbb8f4e805 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:52.761087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1979a26-cbae-4817-b92a-26dbb8f4e805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:52.761611 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1979a26-cbae-4817-b92a-26dbb8f4e805 admin admin] PATH_INFO: `/v3/projects/c6d0fd21d5e64f76b7baefae60befbed` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:52.765844 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1979a26-cbae-4817-b92a-26dbb8f4e805 admin admin] RBAC: Authorizing `identity:get_project(project_id=c6d0fd21d5e64f76b7baefae60befbed)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:52.768591 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1979a26-cbae-4817-b92a-26dbb8f4e805 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:52.774257 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 150/291] 10.222.0.13 () {56 vars in 1207 bytes} [Sun Oct 23 19:52:52 2022] GET /identity/v3/projects/c6d0fd21d5e64f76b7baefae60befbed => generated 344 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:52.776526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1979a26-cbae-4817-b92a-26dbb8f4e805 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:52.794501 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ebcad2-6e10-4f66-a477-595c84d6f971 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:52.799680 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1979a26-cbae-4817-b92a-26dbb8f4e805 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:52.810261 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d2fb7-574e-4632-9035-78612d6903ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:52.811205 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d2fb7-574e-4632-9035-78612d6903ad admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:52.811760 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d2fb7-574e-4632-9035-78612d6903ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:52.813317 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d2fb7-574e-4632-9035-78612d6903ad admin admin] PATH_INFO: `/v3/projects/6da23d78618243288286bfd8d4b7c9d9` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:52.815912 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776d2fb7-574e-4632-9035-78612d6903ad admin admin] RBAC: Authorizing `identity:get_project(project_id=6da23d78618243288286bfd8d4b7c9d9)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:52.819712 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776d2fb7-574e-4632-9035-78612d6903ad admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:52.822533 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 151/292] 10.222.0.13 () {56 vars in 1207 bytes} [Sun Oct 23 19:52:52 2022] GET /identity/v3/projects/6da23d78618243288286bfd8d4b7c9d9 => generated 302 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:52.825853 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d2fb7-574e-4632-9035-78612d6903ad admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:52.847333 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d2fb7-574e-4632-9035-78612d6903ad admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:52.856373 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a7f69d-0fb3-4a03-8fea-52b6c4b7e1f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:52.858621 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7f69d-0fb3-4a03-8fea-52b6c4b7e1f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:52.859899 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7f69d-0fb3-4a03-8fea-52b6c4b7e1f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:52.860413 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a7f69d-0fb3-4a03-8fea-52b6c4b7e1f1 admin admin] PATH_INFO: `/v3/projects/c6d0fd21d5e64f76b7baefae60befbed/groups/a36eacffd2ff41acb073c4d45a457ab9/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:52.864285 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a7f69d-0fb3-4a03-8fea-52b6c4b7e1f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6d0fd21d5e64f76b7baefae60befbed, group_id=a36eacffd2ff41acb073c4d45a457ab9, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:52.866401 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce4221-9456-412a-9d37-93b0d95a2205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:52.867264 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce4221-9456-412a-9d37-93b0d95a2205 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:52.869603 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce4221-9456-412a-9d37-93b0d95a2205 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:52.870087 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce4221-9456-412a-9d37-93b0d95a2205 None admin] PATH_INFO: `/v3/users/dfe31d26b5524db88d5c560c0c44423e` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:52.871597 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ce4221-9456-412a-9d37-93b0d95a2205 None admin] RBAC: Authorizing `identity:get_user(user_id=dfe31d26b5524db88d5c560c0c44423e)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:52.873248 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ce4221-9456-412a-9d37-93b0d95a2205 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:52.874946 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a7f69d-0fb3-4a03-8fea-52b6c4b7e1f1 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:52.880598 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 142/293] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:52:52 2022] GET /identity/v3/users/dfe31d26b5524db88d5c560c0c44423e => generated 293 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:52.882111 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce4221-9456-412a-9d37-93b0d95a2205 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:52.901412 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce4221-9456-412a-9d37-93b0d95a2205 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:52.902064 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 152/294] 10.222.0.13 () {58 vars in 1463 bytes} [Sun Oct 23 19:52:52 2022] PUT /identity/v3/projects/c6d0fd21d5e64f76b7baefae60befbed/groups/a36eacffd2ff41acb073c4d45a457ab9/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 19:52:52.904779 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a7f69d-0fb3-4a03-8fea-52b6c4b7e1f1 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:53.007326 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:53.034819 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19e2801-7eb0-4647-8e5d-a1439cd87d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:53.035505 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e2801-7eb0-4647-8e5d-a1439cd87d44 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:53.035781 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e2801-7eb0-4647-8e5d-a1439cd87d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:53.036050 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19e2801-7eb0-4647-8e5d-a1439cd87d44 admin admin] PATH_INFO: `/v3/projects/6da23d78618243288286bfd8d4b7c9d9/groups/4358694d61434fb99fe9bb5be26fb3a4/roles/2989d29f392d4d8c802354159891eb11` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:53.047200 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19e2801-7eb0-4647-8e5d-a1439cd87d44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6da23d78618243288286bfd8d4b7c9d9, group_id=4358694d61434fb99fe9bb5be26fb3a4, role_id=2989d29f392d4d8c802354159891eb11)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:53.049929 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19e2801-7eb0-4647-8e5d-a1439cd87d44 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:53.056572 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:53.088435 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 143/295] 10.222.0.13 () {58 vars in 1463 bytes} [Sun Oct 23 19:52:52 2022] PUT /identity/v3/projects/6da23d78618243288286bfd8d4b7c9d9/groups/4358694d61434fb99fe9bb5be26fb3a4/roles/2989d29f392d4d8c802354159891eb11 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 19:52:53.123047 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a7f69d-0fb3-4a03-8fea-52b6c4b7e1f1 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:53.181744 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a85c601-08c2-4750-a1fb-fd5252af71f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:53.182796 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85c601-08c2-4750-a1fb-fd5252af71f1 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:53.183316 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85c601-08c2-4750-a1fb-fd5252af71f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:53.183864 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a85c601-08c2-4750-a1fb-fd5252af71f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:53.184889 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a85c601-08c2-4750-a1fb-fd5252af71f1 None admin] RBAC: Adding query filter params (role.id=54768ecdabbd443ca182db572b13b3ca, user.id=dfe31d26b5524db88d5c560c0c44423e, scope.system=all) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:53.185502 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a85c601-08c2-4750-a1fb-fd5252af71f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:53.187890 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a85c601-08c2-4750-a1fb-fd5252af71f1 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:53.210437 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 153/296] 10.222.0.13 () {56 vars in 1308 bytes} [Sun Oct 23 19:52:52 2022] GET /identity/v3/role_assignments?user.id=dfe31d26b5524db88d5c560c0c44423e&scope.system=all&role.id=54768ecdabbd443ca182db572b13b3ca => generated 510 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:53.934332 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49da229-bdba-4aed-b36d-fd2bbe2c4c4c None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:53.934659 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49da229-bdba-4aed-b36d-fd2bbe2c4c4c None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:53.934891 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49da229-bdba-4aed-b36d-fd2bbe2c4c4c None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:53.936277 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 144/297] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:53.944187 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645608f-869d-4b94-8f90-05d56a740cdb None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:53.944652 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645608f-869d-4b94-8f90-05d56a740cdb None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:53.944729 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d645608f-869d-4b94-8f90-05d56a740cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:53.966569 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-d645608f-869d-4b94-8f90-05d56a740cdb None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:53.967772 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d645608f-869d-4b94-8f90-05d56a740cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:54.068151 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 154/298] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:54.135058 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331308df-b50b-4c21-a8a6-10a97888b48a None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:54.135568 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331308df-b50b-4c21-a8a6-10a97888b48a None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:54.135568 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331308df-b50b-4c21-a8a6-10a97888b48a None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:54.136925 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 145/299] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:54.147828 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a32223-a9fb-4996-b19f-4c3619f33ed0 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:54.148115 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a32223-a9fb-4996-b19f-4c3619f33ed0 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:54.148392 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a32223-a9fb-4996-b19f-4c3619f33ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:54.170497 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-52a32223-a9fb-4996-b19f-4c3619f33ed0 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:54.172212 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-52a32223-a9fb-4996-b19f-4c3619f33ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:54.245706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:54.267471 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 155/300] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:54.463192 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331308df-b50b-4c21-a8a6-10a97888b48a None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:54.582980 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331308df-b50b-4c21-a8a6-10a97888b48a None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:54.630982 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a46e1e8-d12c-42bd-9102-2d54e08a119d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:54.631739 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a46e1e8-d12c-42bd-9102-2d54e08a119d None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:54.632027 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a46e1e8-d12c-42bd-9102-2d54e08a119d None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:54.632306 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a46e1e8-d12c-42bd-9102-2d54e08a119d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:54.651442 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a46e1e8-d12c-42bd-9102-2d54e08a119d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:54.657081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-0a46e1e8-d12c-42bd-9102-2d54e08a119d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 19:52:54.657081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:54.657081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:54.657081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:54.657081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:54.657081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:54.657081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:54.657081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:54.657081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:54.657081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:54.657081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:54.657081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:54.657081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:54.657081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:54.657081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:54.657081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:54.657952 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:54.657952 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:54.657952 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:54.657952 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:54.657952 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:54.657952 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:54.657952 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:54.657952 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:54.657952 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:54.657952 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:54.657952 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:54.657952 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:54.657952 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:54.657952 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:54.657952 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:54.657952 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:54.658828 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:54.658828 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:54.658828 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:54.658828 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:54.658828 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:54.658828 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:54.658828 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:54.658828 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:54.658828 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:54.658828 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:54.658828 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 19:52:54.658828 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:52:54.658828 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 146/301] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:52:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:52:54.664593 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a32223-a9fb-4996-b19f-4c3619f33ed0 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:54.782389 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a46e1e8-d12c-42bd-9102-2d54e08a119d None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:54.806190 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a32223-a9fb-4996-b19f-4c3619f33ed0 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:54.845821 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1b4454-cae7-48ff-ba8b-3f7ecb71746e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:54.846800 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b4454-cae7-48ff-ba8b-3f7ecb71746e None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:54.847287 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b4454-cae7-48ff-ba8b-3f7ecb71746e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:54.847761 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1b4454-cae7-48ff-ba8b-3f7ecb71746e None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:54.848623 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1b4454-cae7-48ff-ba8b-3f7ecb71746e None admin] RBAC: Adding query filter params (name=member) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:54.849142 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1b4454-cae7-48ff-ba8b-3f7ecb71746e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:54.850761 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1b4454-cae7-48ff-ba8b-3f7ecb71746e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:54.860558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 156/302] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Oct 23 19:52:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:54.862060 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f6a27-af5c-426c-8ac8-721301a58c9b None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:54.862561 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f6a27-af5c-426c-8ac8-721301a58c9b None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:54.863036 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0f6a27-af5c-426c-8ac8-721301a58c9b None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:54.864537 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 157/303] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:54.865828 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f6a27-af5c-426c-8ac8-721301a58c9b None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:54.869705 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:54.936327 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:54.950383 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a46e1e8-d12c-42bd-9102-2d54e08a119d None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:54.994653 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f1d05-e149-4f3e-9339-bba3a091c042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:54.995697 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f1d05-e149-4f3e-9339-bba3a091c042 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:54.996308 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f1d05-e149-4f3e-9339-bba3a091c042 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:54.996906 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f1d05-e149-4f3e-9339-bba3a091c042 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:55.009504 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0f6a27-af5c-426c-8ac8-721301a58c9b None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:55.026980 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f1d05-e149-4f3e-9339-bba3a091c042 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:55.032640 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-687f1d05-e149-4f3e-9339-bba3a091c042 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 19:52:55.032640 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:55.032640 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:55.032640 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:55.032640 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:55.032640 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:55.032640 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:55.032640 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:55.032640 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:55.032640 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:55.032640 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:55.032640 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:55.032640 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:55.032640 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:55.032640 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:55.032640 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:55.033503 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:55.033503 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:55.033503 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:55.033503 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:55.033503 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:55.033503 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:55.033503 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:55.033503 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:55.033503 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:55.033503 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:55.033503 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:55.033503 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:55.033503 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:55.033503 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:55.033503 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:55.033503 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:55.034397 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:55.034397 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:55.034397 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:55.034397 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:55.034397 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:55.034397 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:55.034397 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:55.034397 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:55.034397 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:55.034397 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:55.034397 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 19:52:55.034397 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:52:55.037495 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 147/304] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Oct 23 19:52:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:52:55.037495 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79179ce-9d9c-4ec1-9fd6-714f75c2dbc5 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:55.037495 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79179ce-9d9c-4ec1-9fd6-714f75c2dbc5 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:55.037861 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79179ce-9d9c-4ec1-9fd6-714f75c2dbc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:55.043596 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b399e-9267-4783-ae44-7b55810f873a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:55.044217 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b399e-9267-4783-ae44-7b55810f873a None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:55.044791 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b399e-9267-4783-ae44-7b55810f873a None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:55.045068 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01b399e-9267-4783-ae44-7b55810f873a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:55.059530 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-f79179ce-9d9c-4ec1-9fd6-714f75c2dbc5 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:55.060726 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-f79179ce-9d9c-4ec1-9fd6-714f75c2dbc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:55.132445 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b399e-9267-4783-ae44-7b55810f873a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:55.134535 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01b399e-9267-4783-ae44-7b55810f873a None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:55.150430 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 148/305] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:55.151375 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79179ce-9d9c-4ec1-9fd6-714f75c2dbc5 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:55.190608 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:55.251348 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 158/306] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Oct 23 19:52:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 23 19:52:55.257884 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b399e-9267-4783-ae44-7b55810f873a None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:55.264369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:55.371511 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01b399e-9267-4783-ae44-7b55810f873a None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:55.375886 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79179ce-9d9c-4ec1-9fd6-714f75c2dbc5 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:55.432943 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870af036-2160-4765-9950-5ad80fecf863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:55.433946 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870af036-2160-4765-9950-5ad80fecf863 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:55.434461 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870af036-2160-4765-9950-5ad80fecf863 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:55.434955 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870af036-2160-4765-9950-5ad80fecf863 None admin] PATH_INFO: `/v3/projects/0a195ad6e19d43ba991afe5458887d8c` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:55.436738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870af036-2160-4765-9950-5ad80fecf863 None admin] RBAC: Authorizing `identity:get_project(project_id=0a195ad6e19d43ba991afe5458887d8c)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:55.439064 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870af036-2160-4765-9950-5ad80fecf863 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:55.443062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 159/307] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Oct 23 19:52:55 2022] GET /identity/v3/projects/0a195ad6e19d43ba991afe5458887d8c => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:55.444445 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d4c405-5838-4366-abe8-e6b7a5521030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:55.445335 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4c405-5838-4366-abe8-e6b7a5521030 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:55.445814 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4c405-5838-4366-abe8-e6b7a5521030 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:55.446311 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d4c405-5838-4366-abe8-e6b7a5521030 None admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:55.447308 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870af036-2160-4765-9950-5ad80fecf863 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:55.448520 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d4c405-5838-4366-abe8-e6b7a5521030 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:55.449384 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d4c405-5838-4366-abe8-e6b7a5521030 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:55.452745 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d4c405-5838-4366-abe8-e6b7a5521030 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:55.468851 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 149/308] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:52:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:55.474490 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d4c405-5838-4366-abe8-e6b7a5521030 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:55.478469 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:55.621416 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870af036-2160-4765-9950-5ad80fecf863 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:55.649662 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d4c405-5838-4366-abe8-e6b7a5521030 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:55.651429 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:55.678980 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d50d54-72f0-401d-8695-4d9eae4e4685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:55.679631 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d50d54-72f0-401d-8695-4d9eae4e4685 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:55.679911 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d50d54-72f0-401d-8695-4d9eae4e4685 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:55.680458 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d50d54-72f0-401d-8695-4d9eae4e4685 None admin] PATH_INFO: `/v3/groups/4358694d61434fb99fe9bb5be26fb3a4` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:55.682204 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d50d54-72f0-401d-8695-4d9eae4e4685 None admin] RBAC: Authorizing `identity:get_group(group_id=4358694d61434fb99fe9bb5be26fb3a4)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:55.684189 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d50d54-72f0-401d-8695-4d9eae4e4685 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:55.686744 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 160/309] 10.222.0.13 () {56 vars in 1180 bytes} [Sun Oct 23 19:52:55 2022] GET /identity/v3/groups/4358694d61434fb99fe9bb5be26fb3a4 => generated 226 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:55.687800 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d50d54-72f0-401d-8695-4d9eae4e4685 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:55.694425 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751649f3-7378-4297-a1a5-cb23566596ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:55.695041 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751649f3-7378-4297-a1a5-cb23566596ea None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:55.695315 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751649f3-7378-4297-a1a5-cb23566596ea None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:55.695582 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751649f3-7378-4297-a1a5-cb23566596ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:55.776078 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751649f3-7378-4297-a1a5-cb23566596ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:55.777536 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751649f3-7378-4297-a1a5-cb23566596ea None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:55.799752 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:55.822184 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d50d54-72f0-401d-8695-4d9eae4e4685 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:55.866136 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701e72f-f427-4583-8f61-83608c096bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:55.866820 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701e72f-f427-4583-8f61-83608c096bb1 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:55.867101 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701e72f-f427-4583-8f61-83608c096bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:55.867378 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3701e72f-f427-4583-8f61-83608c096bb1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:55.878940 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3701e72f-f427-4583-8f61-83608c096bb1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:55.883886 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-3701e72f-f427-4583-8f61-83608c096bb1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 19:52:55.883886 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:55.883886 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:55.883886 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:55.883886 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:55.883886 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:55.883886 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:55.883886 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:55.883886 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:55.883886 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:55.883886 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:55.883886 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:55.883886 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:55.883886 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:55.883886 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:55.883886 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:55.884953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:55.884953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:55.884953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:55.884953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:55.884953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:55.884953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:55.884953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:55.884953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:55.884953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:55.884953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:55.884953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:55.884953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:55.884953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:55.884953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:55.884953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:55.884953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:55.885941 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:55.885941 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:55.885941 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:55.885941 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:55.885941 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:55.885941 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:55.885941 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:55.885941 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:55.885941 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:55.885941 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:55.885941 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 19:52:55.885941 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:52:55.885941 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 161/310] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:52:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:52:55.891154 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701e72f-f427-4583-8f61-83608c096bb1 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:55.893886 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 150/311] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Oct 23 19:52:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 19:52:55.895475 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb0fc5-e857-4f43-81fe-38c6c2e13fe4 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:55.896023 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb0fc5-e857-4f43-81fe-38c6c2e13fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:55.896301 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb0fc5-e857-4f43-81fe-38c6c2e13fe4 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:55.897561 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 151/312] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:55.898278 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb0fc5-e857-4f43-81fe-38c6c2e13fe4 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:55.906084 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:56.023976 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3701e72f-f427-4583-8f61-83608c096bb1 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:56.045627 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb0fc5-e857-4f43-81fe-38c6c2e13fe4 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:56.065969 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cceef674-fc8e-4129-a9ba-ae337fc7be8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:56.066611 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceef674-fc8e-4129-a9ba-ae337fc7be8a None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:56.066886 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceef674-fc8e-4129-a9ba-ae337fc7be8a None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:56.067155 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cceef674-fc8e-4129-a9ba-ae337fc7be8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:56.067882 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceef674-fc8e-4129-a9ba-ae337fc7be8a None admin] RBAC: Adding query filter params (group.id=4358694d61434fb99fe9bb5be26fb3a4, role.id=69610ceae19c4a2b963433e2a9f5bb26, scope.project.id=0a195ad6e19d43ba991afe5458887d8c) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:56.068186 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceef674-fc8e-4129-a9ba-ae337fc7be8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:56.069598 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cceef674-fc8e-4129-a9ba-ae337fc7be8a None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:56.075560 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 162/313] 10.222.0.13 () {56 vars in 1376 bytes} [Sun Oct 23 19:52:55 2022] GET /identity/v3/role_assignments?group.id=4358694d61434fb99fe9bb5be26fb3a4&scope.project.id=0a195ad6e19d43ba991afe5458887d8c&role.id=69610ceae19c4a2b963433e2a9f5bb26 => generated 611 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:56.076736 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7377a43-aed7-4787-b391-7ab66c592e21 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:56.077017 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7377a43-aed7-4787-b391-7ab66c592e21 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:56.077283 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7377a43-aed7-4787-b391-7ab66c592e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:56.092412 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6100456-a527-4ff7-af36-45a76c5c29fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:56.093125 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:56.095732 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6100456-a527-4ff7-af36-45a76c5c29fb None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:56.096027 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6100456-a527-4ff7-af36-45a76c5c29fb None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:56.096344 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6100456-a527-4ff7-af36-45a76c5c29fb None admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:56.097150 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6100456-a527-4ff7-af36-45a76c5c29fb None admin] RBAC: Adding query filter params (name=member) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:56.097467 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6100456-a527-4ff7-af36-45a76c5c29fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:56.099541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6100456-a527-4ff7-af36-45a76c5c29fb None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:56.100795 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-e7377a43-aed7-4787-b391-7ab66c592e21 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:56.102026 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e7377a43-aed7-4787-b391-7ab66c592e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:56.111571 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 152/314] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Oct 23 19:52:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:56.112488 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6100456-a527-4ff7-af36-45a76c5c29fb None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:56.200756 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:56.207562 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 163/315] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:56.208471 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7377a43-aed7-4787-b391-7ab66c592e21 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:56.247441 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6100456-a527-4ff7-af36-45a76c5c29fb None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:56.286940 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ada1ec6-a4f4-4aa5-82c6-e6ebfb60eec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:56.287577 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada1ec6-a4f4-4aa5-82c6-e6ebfb60eec9 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:56.287868 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada1ec6-a4f4-4aa5-82c6-e6ebfb60eec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:56.288130 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ada1ec6-a4f4-4aa5-82c6-e6ebfb60eec9 None admin] PATH_INFO: `/v3/projects/0a195ad6e19d43ba991afe5458887d8c` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:56.289271 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ada1ec6-a4f4-4aa5-82c6-e6ebfb60eec9 None admin] RBAC: Authorizing `identity:get_project(project_id=0a195ad6e19d43ba991afe5458887d8c)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:56.290881 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ada1ec6-a4f4-4aa5-82c6-e6ebfb60eec9 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:56.292514 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 153/316] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Oct 23 19:52:56 2022] GET /identity/v3/projects/0a195ad6e19d43ba991afe5458887d8c => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:56.293622 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e4632-6599-4413-8e2f-202472cd7091 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:56.293897 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e4632-6599-4413-8e2f-202472cd7091 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:56.295271 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4e4632-6599-4413-8e2f-202472cd7091 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:56.297322 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 154/317] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:52:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:52:56.298933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4e4632-6599-4413-8e2f-202472cd7091 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:56.342898 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7377a43-aed7-4787-b391-7ab66c592e21 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:56.388228 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208d457-2c81-432f-88e6-0ebfa5d90dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:56.388846 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208d457-2c81-432f-88e6-0ebfa5d90dc4 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:56.389121 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208d457-2c81-432f-88e6-0ebfa5d90dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:56.389390 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208d457-2c81-432f-88e6-0ebfa5d90dc4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:56.394005 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:56.431767 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4e4632-6599-4413-8e2f-202472cd7091 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:56.469186 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:56.476442 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d208d457-2c81-432f-88e6-0ebfa5d90dc4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:56.481514 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d208d457-2c81-432f-88e6-0ebfa5d90dc4 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:56.481644 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9369a4b8-6a32-46e8-af9e-7127864bb0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:56.481644 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369a4b8-6a32-46e8-af9e-7127864bb0b2 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:56.481644 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369a4b8-6a32-46e8-af9e-7127864bb0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:56.481644 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9369a4b8-6a32-46e8-af9e-7127864bb0b2 None admin] PATH_INFO: `/v3/groups/4358694d61434fb99fe9bb5be26fb3a4` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:56.481927 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9369a4b8-6a32-46e8-af9e-7127864bb0b2 None admin] RBAC: Authorizing `identity:get_group(group_id=4358694d61434fb99fe9bb5be26fb3a4)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:56.484717 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9369a4b8-6a32-46e8-af9e-7127864bb0b2 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:56.487157 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 155/318] 10.222.0.13 () {56 vars in 1180 bytes} [Sun Oct 23 19:52:56 2022] GET /identity/v3/groups/4358694d61434fb99fe9bb5be26fb3a4 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:56.488120 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8482a0-38f8-4d1a-83fb-3af0a77bc6e7 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:56.488446 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8482a0-38f8-4d1a-83fb-3af0a77bc6e7 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:56.488716 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8482a0-38f8-4d1a-83fb-3af0a77bc6e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:56.513903 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-ee8482a0-38f8-4d1a-83fb-3af0a77bc6e7 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:52:56.515146 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-ee8482a0-38f8-4d1a-83fb-3af0a77bc6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:52:56.602310 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 156/319] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:52:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:52:56.602310 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8482a0-38f8-4d1a-83fb-3af0a77bc6e7 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:56.614370 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 164/320] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Oct 23 19:52:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 19:52:56.615384 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208d457-2c81-432f-88e6-0ebfa5d90dc4 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:56.654944 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:56.732097 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8482a0-38f8-4d1a-83fb-3af0a77bc6e7 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:56.738698 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208d457-2c81-432f-88e6-0ebfa5d90dc4 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:56.772487 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:56.781909 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433a24e8-8648-4638-a3f0-4cbe285d649e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:56.782672 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a24e8-8648-4638-a3f0-4cbe285d649e None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:56.782963 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a24e8-8648-4638-a3f0-4cbe285d649e None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:56.783284 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a24e8-8648-4638-a3f0-4cbe285d649e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:56.783583 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae16f5a-f66e-41db-9bf1-1b791c2aadb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:56.784038 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433a24e8-8648-4638-a3f0-4cbe285d649e None admin] RBAC: Adding query filter params (group.id=4358694d61434fb99fe9bb5be26fb3a4, role.id=2989d29f392d4d8c802354159891eb11, scope.project.id=0a195ad6e19d43ba991afe5458887d8c) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:56.784213 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae16f5a-f66e-41db-9bf1-1b791c2aadb5 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:56.784358 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433a24e8-8648-4638-a3f0-4cbe285d649e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:56.784504 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae16f5a-f66e-41db-9bf1-1b791c2aadb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:56.784758 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae16f5a-f66e-41db-9bf1-1b791c2aadb5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:56.787011 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433a24e8-8648-4638-a3f0-4cbe285d649e None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:56.793915 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 157/321] 10.222.0.13 () {56 vars in 1376 bytes} [Sun Oct 23 19:52:56 2022] GET /identity/v3/role_assignments?group.id=4358694d61434fb99fe9bb5be26fb3a4&scope.project.id=0a195ad6e19d43ba991afe5458887d8c&role.id=2989d29f392d4d8c802354159891eb11 => generated 611 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:56.795451 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae16f5a-f66e-41db-9bf1-1b791c2aadb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:56.800240 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433a24e8-8648-4638-a3f0-4cbe285d649e None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:56.802015 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-2ae16f5a-f66e-41db-9bf1-1b791c2aadb5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:52:56.802015 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:56.802015 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:56.802015 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:56.802015 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:56.802015 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:56.802015 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:56.802015 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:56.802015 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:56.802015 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:56.802015 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:56.802015 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:56.802015 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:56.802015 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:56.802015 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:56.802015 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:56.802968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:56.802968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:56.802968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:56.802968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:56.802968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:56.802968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:56.802968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:56.802968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:56.802968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:56.802968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:56.802968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:56.802968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:56.802968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:56.802968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:56.802968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:56.802968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:56.803764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:56.803764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:56.803764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:56.803764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:56.803764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:56.803764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:56.803764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:56.803764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:56.803764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:56.803764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:56.803764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:52:56.803764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:52:56.803764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 165/322] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Oct 23 19:52:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:52:56.809594 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae16f5a-f66e-41db-9bf1-1b791c2aadb5 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:56.939747 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:56.940444 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433a24e8-8648-4638-a3f0-4cbe285d649e None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:56.945475 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae16f5a-f66e-41db-9bf1-1b791c2aadb5 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:56.983343 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff4a20-b4ed-4172-bb24-7930b1e2fb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:56.984067 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff4a20-b4ed-4172-bb24-7930b1e2fb09 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:56.984363 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff4a20-b4ed-4172-bb24-7930b1e2fb09 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:56.984629 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eff4a20-b4ed-4172-bb24-7930b1e2fb09 None admin] PATH_INFO: `/v3/projects/6da23d78618243288286bfd8d4b7c9d9` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:56.985934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eff4a20-b4ed-4172-bb24-7930b1e2fb09 None admin] RBAC: Authorizing `identity:get_project(project_id=6da23d78618243288286bfd8d4b7c9d9)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:56.987730 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eff4a20-b4ed-4172-bb24-7930b1e2fb09 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:56.989746 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 158/323] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Oct 23 19:52:56 2022] GET /identity/v3/projects/6da23d78618243288286bfd8d4b7c9d9 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:56.990497 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff4a20-b4ed-4172-bb24-7930b1e2fb09 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:56.992112 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5e2d0a-dfed-46ab-9e80-e7f0b337cdc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:56.992728 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5e2d0a-dfed-46ab-9e80-e7f0b337cdc3 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:56.993002 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5e2d0a-dfed-46ab-9e80-e7f0b337cdc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:56.993268 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5e2d0a-dfed-46ab-9e80-e7f0b337cdc3 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:56.994004 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5e2d0a-dfed-46ab-9e80-e7f0b337cdc3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:56.994337 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5e2d0a-dfed-46ab-9e80-e7f0b337cdc3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:56.995856 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5e2d0a-dfed-46ab-9e80-e7f0b337cdc3 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:57.007378 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 166/324] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:52:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:57.008233 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5e2d0a-dfed-46ab-9e80-e7f0b337cdc3 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:57.070095 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:57.119965 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eff4a20-b4ed-4172-bb24-7930b1e2fb09 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:57.132326 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5e2d0a-dfed-46ab-9e80-e7f0b337cdc3 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:57.158997 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686629da-3a3b-4ce4-9a6a-b91924f1de0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:57.159636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686629da-3a3b-4ce4-9a6a-b91924f1de0f None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:57.159918 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686629da-3a3b-4ce4-9a6a-b91924f1de0f None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:57.160190 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686629da-3a3b-4ce4-9a6a-b91924f1de0f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:57.170336 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cceff7a-2bb6-43d2-bfc1-a8eff254842a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:57.170981 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cceff7a-2bb6-43d2-bfc1-a8eff254842a None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:57.171260 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cceff7a-2bb6-43d2-bfc1-a8eff254842a None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:57.171572 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cceff7a-2bb6-43d2-bfc1-a8eff254842a None admin] PATH_INFO: `/v3/groups/4358694d61434fb99fe9bb5be26fb3a4` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:57.171750 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686629da-3a3b-4ce4-9a6a-b91924f1de0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:52:57.172922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cceff7a-2bb6-43d2-bfc1-a8eff254842a None admin] RBAC: Authorizing `identity:get_group(group_id=4358694d61434fb99fe9bb5be26fb3a4)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:57.174134 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cceff7a-2bb6-43d2-bfc1-a8eff254842a None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:57.176000 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 167/325] 10.222.0.13 () {56 vars in 1180 bytes} [Sun Oct 23 19:52:57 2022] GET /identity/v3/groups/4358694d61434fb99fe9bb5be26fb3a4 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:57.177374 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-686629da-3a3b-4ce4-9a6a-b91924f1de0f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 19:52:57.177374 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:52:57.177374 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:52:57.177374 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:52:57.177374 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:52:57.177374 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:52:57.177374 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:52:57.177374 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:52:57.177374 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:52:57.177374 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:52:57.177374 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:52:57.177374 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:52:57.177374 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:52:57.177374 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:52:57.177374 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:52:57.177374 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:52:57.178212 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:52:57.178212 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:52:57.178212 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:52:57.178212 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:52:57.178212 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:52:57.178212 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:52:57.178212 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:52:57.178212 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:52:57.178212 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:52:57.178212 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:52:57.178212 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:52:57.178212 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:52:57.178212 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:52:57.178212 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:52:57.178212 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:52:57.178212 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:52:57.179037 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:52:57.179037 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:52:57.179037 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:52:57.179037 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:52:57.179037 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:52:57.179037 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:52:57.179037 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:52:57.179037 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:52:57.179037 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:52:57.179037 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:52:57.179037 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 19:52:57.179037 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:52:57.179653 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cceff7a-2bb6-43d2-bfc1-a8eff254842a None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:57.179720 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 159/326] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Oct 23 19:52:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:52:57.183950 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686629da-3a3b-4ce4-9a6a-b91924f1de0f None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:57.216422 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:57.303605 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cceff7a-2bb6-43d2-bfc1-a8eff254842a None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:57.310564 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686629da-3a3b-4ce4-9a6a-b91924f1de0f None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:57.333812 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:57.345012 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9fa9cb-0d3c-4636-90ab-800e9a029d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:57.346283 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fa9cb-0d3c-4636-90ab-800e9a029d1a None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:57.346283 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fa9cb-0d3c-4636-90ab-800e9a029d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:57.346283 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9fa9cb-0d3c-4636-90ab-800e9a029d1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:57.348684 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1934a79-c1b7-45bc-aa78-ba574fecac25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:57.350822 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1934a79-c1b7-45bc-aa78-ba574fecac25 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:57.350822 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1934a79-c1b7-45bc-aa78-ba574fecac25 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:57.350822 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1934a79-c1b7-45bc-aa78-ba574fecac25 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:57.350822 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1934a79-c1b7-45bc-aa78-ba574fecac25 None admin] RBAC: Adding query filter params (group.id=4358694d61434fb99fe9bb5be26fb3a4, role.id=69610ceae19c4a2b963433e2a9f5bb26, scope.project.id=6da23d78618243288286bfd8d4b7c9d9) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:57.351121 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1934a79-c1b7-45bc-aa78-ba574fecac25 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:57.352536 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1934a79-c1b7-45bc-aa78-ba574fecac25 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:57.360703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 160/327] 10.222.0.13 () {56 vars in 1376 bytes} [Sun Oct 23 19:52:57 2022] GET /identity/v3/role_assignments?group.id=4358694d61434fb99fe9bb5be26fb3a4&scope.project.id=6da23d78618243288286bfd8d4b7c9d9&role.id=69610ceae19c4a2b963433e2a9f5bb26 => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:57.360703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1934a79-c1b7-45bc-aa78-ba574fecac25 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:57.445256 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9fa9cb-0d3c-4636-90ab-800e9a029d1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:57.446785 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9fa9cb-0d3c-4636-90ab-800e9a029d1a None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:57.478732 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:57.493655 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1934a79-c1b7-45bc-aa78-ba574fecac25 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:57.532443 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52903fa6-50b8-49cc-9f32-41cdafdaf482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:57.533058 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52903fa6-50b8-49cc-9f32-41cdafdaf482 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:57.533334 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52903fa6-50b8-49cc-9f32-41cdafdaf482 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:57.533602 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52903fa6-50b8-49cc-9f32-41cdafdaf482 None admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:57.534422 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52903fa6-50b8-49cc-9f32-41cdafdaf482 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:57.534598 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52903fa6-50b8-49cc-9f32-41cdafdaf482 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:57.535967 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52903fa6-50b8-49cc-9f32-41cdafdaf482 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:57.545200 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 161/328] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:52:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:52:57.551265 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52903fa6-50b8-49cc-9f32-41cdafdaf482 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:57.564583 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 168/329] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Oct 23 19:52:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 19:52:57.570500 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9fa9cb-0d3c-4636-90ab-800e9a029d1a None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:57.608550 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:57.666462 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52903fa6-50b8-49cc-9f32-41cdafdaf482 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:57.689505 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9fa9cb-0d3c-4636-90ab-800e9a029d1a None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:57.707847 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c470615d-e231-401e-b71d-a1cf1a698107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:57.708470 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470615d-e231-401e-b71d-a1cf1a698107 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:57.708751 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470615d-e231-401e-b71d-a1cf1a698107 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:57.709026 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c470615d-e231-401e-b71d-a1cf1a698107 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:57.723590 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:57.735067 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be84ac-5eeb-4ff9-aeb5-86b45642ede8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:57.735732 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be84ac-5eeb-4ff9-aeb5-86b45642ede8 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:57.736085 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be84ac-5eeb-4ff9-aeb5-86b45642ede8 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:57.736428 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be84ac-5eeb-4ff9-aeb5-86b45642ede8 None admin] PATH_INFO: `/v3/projects/c6d0fd21d5e64f76b7baefae60befbed` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:57.737647 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be84ac-5eeb-4ff9-aeb5-86b45642ede8 None admin] RBAC: Authorizing `identity:get_project(project_id=c6d0fd21d5e64f76b7baefae60befbed)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:57.739135 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be84ac-5eeb-4ff9-aeb5-86b45642ede8 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:57.740969 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 169/330] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Oct 23 19:52:57 2022] GET /identity/v3/projects/c6d0fd21d5e64f76b7baefae60befbed => generated 344 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:57.746199 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be84ac-5eeb-4ff9-aeb5-86b45642ede8 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:57.783784 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c470615d-e231-401e-b71d-a1cf1a698107 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:57.785225 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c470615d-e231-401e-b71d-a1cf1a698107 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:57.832064 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:57.848754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be84ac-5eeb-4ff9-aeb5-86b45642ede8 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:57.882606 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7736022-3e84-4744-b5b9-0107d602d2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:57.882790 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 162/331] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Oct 23 19:52:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 19:52:57.883230 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7736022-3e84-4744-b5b9-0107d602d2ef None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:57.883507 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7736022-3e84-4744-b5b9-0107d602d2ef None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:57.883772 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7736022-3e84-4744-b5b9-0107d602d2ef None admin] PATH_INFO: `/v3/groups/a36eacffd2ff41acb073c4d45a457ab9` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:57.884865 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7736022-3e84-4744-b5b9-0107d602d2ef None admin] RBAC: Authorizing `identity:get_group(group_id=a36eacffd2ff41acb073c4d45a457ab9)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:57.886300 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7736022-3e84-4744-b5b9-0107d602d2ef None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:57.887858 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 170/332] 10.222.0.13 () {56 vars in 1180 bytes} [Sun Oct 23 19:52:57 2022] GET /identity/v3/groups/a36eacffd2ff41acb073c4d45a457ab9 => generated 229 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:57.888223 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c470615d-e231-401e-b71d-a1cf1a698107 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:57.893965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7736022-3e84-4744-b5b9-0107d602d2ef None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:57.955001 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:57.984819 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c470615d-e231-401e-b71d-a1cf1a698107 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:57.993118 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7736022-3e84-4744-b5b9-0107d602d2ef None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:58.018652 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36b6f3-7a96-43d7-86ac-1c85ed94311d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:58.019288 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36b6f3-7a96-43d7-86ac-1c85ed94311d None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:58.019562 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36b6f3-7a96-43d7-86ac-1c85ed94311d None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:58.019831 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a36b6f3-7a96-43d7-86ac-1c85ed94311d None admin] PATH_INFO: `/v3/projects/6da23d78618243288286bfd8d4b7c9d9` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:58.020942 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a36b6f3-7a96-43d7-86ac-1c85ed94311d None admin] RBAC: Authorizing `identity:get_project(project_id=6da23d78618243288286bfd8d4b7c9d9)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:58.022405 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a36b6f3-7a96-43d7-86ac-1c85ed94311d None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:58.024030 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 163/333] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Oct 23 19:52:57 2022] GET /identity/v3/projects/6da23d78618243288286bfd8d4b7c9d9 => generated 302 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:58.029251 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36b6f3-7a96-43d7-86ac-1c85ed94311d None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:58.029961 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ea6c7-849e-4691-90b3-47d02be0f90b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:58.030594 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ea6c7-849e-4691-90b3-47d02be0f90b None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:58.030927 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ea6c7-849e-4691-90b3-47d02be0f90b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:58.031207 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ea6c7-849e-4691-90b3-47d02be0f90b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:58.031940 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ea6c7-849e-4691-90b3-47d02be0f90b None admin] RBAC: Adding query filter params (group.id=a36eacffd2ff41acb073c4d45a457ab9, role.id=742dfaf63ef74207a2c9213411513925, scope.project.id=c6d0fd21d5e64f76b7baefae60befbed) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:58.032249 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ea6c7-849e-4691-90b3-47d02be0f90b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:58.034078 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7ea6c7-849e-4691-90b3-47d02be0f90b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:58.040671 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 171/334] 10.222.0.13 () {56 vars in 1376 bytes} [Sun Oct 23 19:52:57 2022] GET /identity/v3/role_assignments?group.id=a36eacffd2ff41acb073c4d45a457ab9&scope.project.id=c6d0fd21d5e64f76b7baefae60befbed&role.id=742dfaf63ef74207a2c9213411513925 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:58.053702 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:58.142082 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a36b6f3-7a96-43d7-86ac-1c85ed94311d None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:58.176849 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f394b1-56a6-4a60-b41c-210bf7151263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:58.177485 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f394b1-56a6-4a60-b41c-210bf7151263 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:58.177776 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f394b1-56a6-4a60-b41c-210bf7151263 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:58.178035 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f394b1-56a6-4a60-b41c-210bf7151263 None admin] PATH_INFO: `/v3/groups/4358694d61434fb99fe9bb5be26fb3a4` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:58.179191 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f394b1-56a6-4a60-b41c-210bf7151263 None admin] RBAC: Authorizing `identity:get_group(group_id=4358694d61434fb99fe9bb5be26fb3a4)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:58.180631 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f394b1-56a6-4a60-b41c-210bf7151263 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:58.182288 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 164/335] 10.222.0.13 () {56 vars in 1180 bytes} [Sun Oct 23 19:52:58 2022] GET /identity/v3/groups/4358694d61434fb99fe9bb5be26fb3a4 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:52:58.188993 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ea6c7-849e-4691-90b3-47d02be0f90b None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:52:58.290188 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7ea6c7-849e-4691-90b3-47d02be0f90b None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:52:58.315465 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:52:58.324253 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b688529c-bd1f-45d8-9d9c-e3a524c90093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:52:58.324854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688529c-bd1f-45d8-9d9c-e3a524c90093 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:52:58.325136 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688529c-bd1f-45d8-9d9c-e3a524c90093 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:52:58.325413 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b688529c-bd1f-45d8-9d9c-e3a524c90093 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:52:58.326139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b688529c-bd1f-45d8-9d9c-e3a524c90093 None admin] RBAC: Adding query filter params (group.id=4358694d61434fb99fe9bb5be26fb3a4, role.id=2989d29f392d4d8c802354159891eb11, scope.project.id=6da23d78618243288286bfd8d4b7c9d9) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:52:58.326721 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b688529c-bd1f-45d8-9d9c-e3a524c90093 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:52:58.328201 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b688529c-bd1f-45d8-9d9c-e3a524c90093 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:52:58.333855 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 172/336] 10.222.0.13 () {56 vars in 1376 bytes} [Sun Oct 23 19:52:58 2022] GET /identity/v3/role_assignments?group.id=4358694d61434fb99fe9bb5be26fb3a4&scope.project.id=6da23d78618243288286bfd8d4b7c9d9&role.id=2989d29f392d4d8c802354159891eb11 => generated 611 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:00.922531 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef07b0-ef1f-4291-b969-582e93fe50a3 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:00.923154 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef07b0-ef1f-4291-b969-582e93fe50a3 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:00.923154 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef07b0-ef1f-4291-b969-582e93fe50a3 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:00.924685 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 165/337] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:00.930088 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cacaa5-437a-46e6-8256-391b71cdb086 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:00.930527 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cacaa5-437a-46e6-8256-391b71cdb086 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:00.930694 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cacaa5-437a-46e6-8256-391b71cdb086 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:00.953762 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-f6cacaa5-437a-46e6-8256-391b71cdb086 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:00.954973 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f6cacaa5-437a-46e6-8256-391b71cdb086 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:01.034907 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 173/338] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 19:53:01.273946 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc27687-ad84-4ee8-b689-212a8ad5a25e None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:01.274126 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc27687-ad84-4ee8-b689-212a8ad5a25e None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:01.274452 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc27687-ad84-4ee8-b689-212a8ad5a25e None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:01.276134 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 166/339] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:01.289445 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f4acf0-e2b7-4e0c-9638-496464ec4031 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:01.289725 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f4acf0-e2b7-4e0c-9638-496464ec4031 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:01.290009 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f4acf0-e2b7-4e0c-9638-496464ec4031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:01.318830 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-b5f4acf0-e2b7-4e0c-9638-496464ec4031 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:01.320028 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-b5f4acf0-e2b7-4e0c-9638-496464ec4031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:01.348385 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc27687-ad84-4ee8-b689-212a8ad5a25e None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:01.431114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 174/340] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:53:01.474837 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:01.493049 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc27687-ad84-4ee8-b689-212a8ad5a25e None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:01.547068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c427b831-2283-420f-85b9-1ea0b1c0c6f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:01.547623 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427b831-2283-420f-85b9-1ea0b1c0c6f3 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:01.547902 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427b831-2283-420f-85b9-1ea0b1c0c6f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:01.548172 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427b831-2283-420f-85b9-1ea0b1c0c6f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:01.561466 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c427b831-2283-420f-85b9-1ea0b1c0c6f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:01.563115 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c427b831-2283-420f-85b9-1ea0b1c0c6f3 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:01.584372 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-c427b831-2283-420f-85b9-1ea0b1c0c6f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:01.584372 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:01.584372 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:01.584372 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:01.584372 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:01.584372 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:01.584372 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:01.584372 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:01.584372 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:01.584372 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:01.584372 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:01.584372 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:01.584372 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:01.584372 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:01.584372 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:01.584372 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:01.585380 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:01.585380 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:01.585380 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:01.585380 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:01.585380 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:01.585380 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:01.585380 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:01.585380 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:01.585380 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:01.585380 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:01.585380 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:01.585380 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:01.585380 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:01.585380 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:01.585380 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:01.585380 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:01.585380 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:01.587338 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:01.587338 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:01.587338 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:01.587338 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 167/341] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:01.591878 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f4acf0-e2b7-4e0c-9638-496464ec4031 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:01.628385 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5535c-92ac-49d1-9c71-5ce509a98ee6 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:01.628694 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5535c-92ac-49d1-9c71-5ce509a98ee6 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:01.628937 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c5535c-92ac-49d1-9c71-5ce509a98ee6 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:01.631206 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 168/342] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:01.636132 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d687d-45a7-4055-8782-1c5bfe0c99a0 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:01.636578 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d687d-45a7-4055-8782-1c5bfe0c99a0 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:01.636949 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d687d-45a7-4055-8782-1c5bfe0c99a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:01.662675 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-3c2d687d-45a7-4055-8782-1c5bfe0c99a0 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:01.664085 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-3c2d687d-45a7-4055-8782-1c5bfe0c99a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:01.689575 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:01.770438 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f4acf0-e2b7-4e0c-9638-496464ec4031 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:01.771191 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 169/343] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:53:01.772499 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b90f8-06b4-4428-804b-01f0771f2a29 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:01.772823 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b90f8-06b4-4428-804b-01f0771f2a29 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:01.773094 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238b90f8-06b4-4428-804b-01f0771f2a29 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:01.778201 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 170/344] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:01.779620 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd2818-05e0-4e80-b396-f6add613f84d None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:01.779902 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd2818-05e0-4e80-b396-f6add613f84d None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:01.780174 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dd2818-05e0-4e80-b396-f6add613f84d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:01.798756 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-f7dd2818-05e0-4e80-b396-f6add613f84d None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:01.799853 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-f7dd2818-05e0-4e80-b396-f6add613f84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:01.833820 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ead4dc-dd4a-4ccb-a190-85334943ec60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:01.834796 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ead4dc-dd4a-4ccb-a190-85334943ec60 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:01.835350 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ead4dc-dd4a-4ccb-a190-85334943ec60 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:01.835842 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ead4dc-dd4a-4ccb-a190-85334943ec60 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:01.837195 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ead4dc-dd4a-4ccb-a190-85334943ec60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:01.838621 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ead4dc-dd4a-4ccb-a190-85334943ec60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:01.840218 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ead4dc-dd4a-4ccb-a190-85334943ec60 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:01.855831 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 175/345] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:01.857558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ead4dc-dd4a-4ccb-a190-85334943ec60 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:01.864847 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:01.876225 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 171/346] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 19:53:01.877262 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd2818-05e0-4e80-b396-f6add613f84d None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:02.000227 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ead4dc-dd4a-4ccb-a190-85334943ec60 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:02.012915 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:02.059840 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d82f09-1f20-4bd3-b852-6f4a04c929ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:02.060462 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d82f09-1f20-4bd3-b852-6f4a04c929ca None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:02.060741 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d82f09-1f20-4bd3-b852-6f4a04c929ca None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:02.061188 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d82f09-1f20-4bd3-b852-6f4a04c929ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:02.073417 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d82f09-1f20-4bd3-b852-6f4a04c929ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:02.075119 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d82f09-1f20-4bd3-b852-6f4a04c929ca None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:02.083800 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dd2818-05e0-4e80-b396-f6add613f84d None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:02.093114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-f2d82f09-1f20-4bd3-b852-6f4a04c929ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:02.093114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:02.093114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:02.093114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:02.093114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:02.093114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:02.093114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:02.093114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:02.093114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:02.093114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:02.093114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:02.093114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:02.093114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:02.093114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:02.093114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:02.093114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:02.094133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:02.094133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:02.094133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:02.094133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:02.094133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:02.094133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:02.094133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:02.094133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:02.094133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:02.094133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:02.094133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:02.094133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:02.094133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:02.094133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:02.094133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:02.094133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:02.094133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:02.095131 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:02.095131 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:02.095131 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:02.096593 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 176/347] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:02.097529 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d82f09-1f20-4bd3-b852-6f4a04c929ca None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:02.122215 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6925215-8200-479b-b4af-0870bbe195dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:02.122984 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6925215-8200-479b-b4af-0870bbe195dd None admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:02.123271 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6925215-8200-479b-b4af-0870bbe195dd None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:02.123546 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6925215-8200-479b-b4af-0870bbe195dd None admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:02.124296 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6925215-8200-479b-b4af-0870bbe195dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:02.126304 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6925215-8200-479b-b4af-0870bbe195dd None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:02.149567 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 172/348] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Oct 23 19:53:01 2022] POST /identity/v3/users => generated 246 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:53:02.150508 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6925215-8200-479b-b4af-0870bbe195dd None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:02.247499 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d82f09-1f20-4bd3-b852-6f4a04c929ca None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:02.264924 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:02.294890 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94146fd7-6960-4a19-b38d-a8ded15098f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:02.295560 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94146fd7-6960-4a19-b38d-a8ded15098f3 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:02.295837 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94146fd7-6960-4a19-b38d-a8ded15098f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:02.296099 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94146fd7-6960-4a19-b38d-a8ded15098f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:02.310515 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94146fd7-6960-4a19-b38d-a8ded15098f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:02.311874 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94146fd7-6960-4a19-b38d-a8ded15098f3 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:02.322981 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6925215-8200-479b-b4af-0870bbe195dd None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:02.333214 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-94146fd7-6960-4a19-b38d-a8ded15098f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:02.333214 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:02.333214 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:02.333214 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:02.333214 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:02.333214 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:02.333214 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:02.333214 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:02.333214 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:02.333214 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:02.333214 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:02.333214 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:02.333214 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:02.333214 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:02.333214 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:02.333214 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:02.334202 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:02.334202 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:02.334202 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:02.334202 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:02.334202 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:02.334202 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:02.334202 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:02.334202 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:02.334202 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:02.334202 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:02.334202 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:02.334202 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:02.334202 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:02.334202 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:02.334202 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:02.334202 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:02.334202 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:02.335072 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:02.335072 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:02.335072 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:02.335072 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 177/349] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:02.335286 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94146fd7-6960-4a19-b38d-a8ded15098f3 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:02.364135 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:02.377504 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a367d7-8d07-4bf1-8e97-15ffba385aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:02.378284 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a367d7-8d07-4bf1-8e97-15ffba385aec None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:02.378628 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a367d7-8d07-4bf1-8e97-15ffba385aec None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:02.378972 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a367d7-8d07-4bf1-8e97-15ffba385aec None admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:02.379684 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a367d7-8d07-4bf1-8e97-15ffba385aec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:02.380067 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a367d7-8d07-4bf1-8e97-15ffba385aec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:02.381505 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a367d7-8d07-4bf1-8e97-15ffba385aec None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:02.401745 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 173/350] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:02.402933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a367d7-8d07-4bf1-8e97-15ffba385aec None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:02.463253 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94146fd7-6960-4a19-b38d-a8ded15098f3 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:02.498499 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c7dea-edaa-4b1a-a11a-8a1dc0c73e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:02.499229 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c7dea-edaa-4b1a-a11a-8a1dc0c73e91 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:02.500137 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c7dea-edaa-4b1a-a11a-8a1dc0c73e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:02.500510 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81c7dea-edaa-4b1a-a11a-8a1dc0c73e91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:02.512621 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81c7dea-edaa-4b1a-a11a-8a1dc0c73e91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:02.513878 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81c7dea-edaa-4b1a-a11a-8a1dc0c73e91 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:02.525348 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a367d7-8d07-4bf1-8e97-15ffba385aec None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:02.529534 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-f81c7dea-edaa-4b1a-a11a-8a1dc0c73e91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:02.529534 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:02.529534 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:02.529534 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:02.529534 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:02.529534 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:02.529534 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:02.529534 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:02.529534 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:02.529534 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:02.529534 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:02.529534 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:02.529534 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:02.529534 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:02.529534 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:02.529534 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:02.530445 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:02.530445 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:02.530445 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:02.530445 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:02.530445 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:02.530445 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:02.530445 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:02.530445 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:02.530445 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:02.530445 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:02.530445 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:02.530445 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:02.530445 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:02.530445 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:02.530445 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:02.530445 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:02.530445 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:02.531330 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:02.531330 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:02.531330 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:02.531330 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 178/351] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:02.531816 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c7dea-edaa-4b1a-a11a-8a1dc0c73e91 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:02.542293 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:02.566377 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e4680-d47e-48e5-aa42-7656db837997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:02.567470 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e4680-d47e-48e5-aa42-7656db837997 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:02.567941 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e4680-d47e-48e5-aa42-7656db837997 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:02.568397 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e4680-d47e-48e5-aa42-7656db837997 None admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:02.569232 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e4680-d47e-48e5-aa42-7656db837997 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:02.569768 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e4680-d47e-48e5-aa42-7656db837997 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:02.571443 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e4680-d47e-48e5-aa42-7656db837997 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:02.584310 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 174/352] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:02.585416 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e4680-d47e-48e5-aa42-7656db837997 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:02.615440 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:02.654566 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81c7dea-edaa-4b1a-a11a-8a1dc0c73e91 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:02.692556 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf27f9-7ed3-4b25-8833-8c1d7f1b6fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:02.693169 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf27f9-7ed3-4b25-8833-8c1d7f1b6fb7 None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:02.693514 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf27f9-7ed3-4b25-8833-8c1d7f1b6fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:02.693890 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf27f9-7ed3-4b25-8833-8c1d7f1b6fb7 None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:02.694658 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bf27f9-7ed3-4b25-8833-8c1d7f1b6fb7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:02.696251 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bf27f9-7ed3-4b25-8833-8c1d7f1b6fb7 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:02.696508 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e4680-d47e-48e5-aa42-7656db837997 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:02.725465 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 179/353] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Oct 23 19:53:02 2022] POST /identity/v3/users => generated 248 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:53:02.725465 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf27f9-7ed3-4b25-8833-8c1d7f1b6fb7 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:02.759340 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578838c-1fec-40b1-9f24-006962a7870f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:02.759974 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578838c-1fec-40b1-9f24-006962a7870f None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:02.760253 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578838c-1fec-40b1-9f24-006962a7870f None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:02.760525 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a578838c-1fec-40b1-9f24-006962a7870f None admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:02.761204 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a578838c-1fec-40b1-9f24-006962a7870f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:02.761505 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a578838c-1fec-40b1-9f24-006962a7870f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:02.764436 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a578838c-1fec-40b1-9f24-006962a7870f None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:02.793745 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 175/354] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:02.803235 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578838c-1fec-40b1-9f24-006962a7870f None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:02.866665 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:02.914411 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf27f9-7ed3-4b25-8833-8c1d7f1b6fb7 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:02.943822 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:02.945698 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a578838c-1fec-40b1-9f24-006962a7870f None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:02.957611 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3be6e7c-cfe8-414b-b70d-93c0473bdc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:02.958978 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be6e7c-cfe8-414b-b70d-93c0473bdc99 None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:02.959564 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be6e7c-cfe8-414b-b70d-93c0473bdc99 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:02.960005 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3be6e7c-cfe8-414b-b70d-93c0473bdc99 None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:02.960815 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3be6e7c-cfe8-414b-b70d-93c0473bdc99 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:02.962659 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3be6e7c-cfe8-414b-b70d-93c0473bdc99 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:02.988766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 180/355] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Oct 23 19:53:02 2022] POST /identity/v3/users => generated 248 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:53:02.991027 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61bd5f-250d-4caa-a11c-bdceeeaef38a None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:02.991471 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61bd5f-250d-4caa-a11c-bdceeeaef38a None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:02.991923 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61bd5f-250d-4caa-a11c-bdceeeaef38a None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:02.993429 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 181/356] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:02.996366 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f530c69-8def-48d9-bc53-6d9e8e956469 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:02.997231 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f530c69-8def-48d9-bc53-6d9e8e956469 None admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:02.997642 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f530c69-8def-48d9-bc53-6d9e8e956469 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:02.998062 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f530c69-8def-48d9-bc53-6d9e8e956469 None admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:03.000263 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b547f5-1deb-43fb-aec0-5e419f7d1ef5 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:03.000729 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b547f5-1deb-43fb-aec0-5e419f7d1ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:03.000802 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b547f5-1deb-43fb-aec0-5e419f7d1ef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:03.003269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f530c69-8def-48d9-bc53-6d9e8e956469 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:03.005029 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f530c69-8def-48d9-bc53-6d9e8e956469 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:03.024038 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-17b547f5-1deb-43fb-aec0-5e419f7d1ef5 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:03.025722 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-17b547f5-1deb-43fb-aec0-5e419f7d1ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:03.039118 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 176/357] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Oct 23 19:53:02 2022] POST /identity/v3/users => generated 249 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:53:03.138961 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 182/358] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 19:53:03.421872 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f530c69-8def-48d9-bc53-6d9e8e956469 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:03.537359 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:03.550783 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f530c69-8def-48d9-bc53-6d9e8e956469 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:03.591186 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b2473-3646-4d1a-b079-ca13fab62a24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:03.592180 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b2473-3646-4d1a-b079-ca13fab62a24 None admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:03.592691 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b2473-3646-4d1a-b079-ca13fab62a24 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:03.593158 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b2473-3646-4d1a-b079-ca13fab62a24 None admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:03.594145 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5b2473-3646-4d1a-b079-ca13fab62a24 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:03.595759 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5b2473-3646-4d1a-b079-ca13fab62a24 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:03.604976 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.sql.core [None req-ed5b2473-3646-4d1a-b079-ca13fab62a24 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 23 19:53:03.604976 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 23 19:53:03.604976 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [parameters: {'id': 'e421c7eaf04f4bf99b8c550814f95194', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 23 19:53:03.604976 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81412) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 23 19:53:03.622531 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-ed5b2473-3646-4d1a-b079-ca13fab62a24 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 23 19:53:03.622531 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:03.622531 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 23 19:53:03.622531 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 23 19:53:03.622531 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 23 19:53:03.622531 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 23 19:53:03.622531 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 23 19:53:03.622531 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application result = self._query(query) Oct 23 19:53:03.622531 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 23 19:53:03.622531 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application conn.query(q) Oct 23 19:53:03.622531 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 23 19:53:03.622531 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 23 19:53:03.622531 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 23 19:53:03.622531 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application result.read() Oct 23 19:53:03.622531 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 23 19:53:03.623486 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 23 19:53:03.623486 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 23 19:53:03.623486 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 23 19:53:03.623486 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 23 19:53:03.623486 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 23 19:53:03.623486 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 23 19:53:03.623486 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 23 19:53:03.623486 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 23 19:53:03.623486 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:03.623486 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 23 19:53:03.623486 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:03.623486 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:03.623486 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 23 19:53:03.623486 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 23 19:53:03.623486 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 23 19:53:03.623486 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return ref.to_dict() Oct 23 19:53:03.623486 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 23 19:53:03.623486 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application next(self.gen) Oct 23 19:53:03.624435 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 23 19:53:03.624435 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application yield resource Oct 23 19:53:03.624435 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 23 19:53:03.624435 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application next(self.gen) Oct 23 19:53:03.624435 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 23 19:53:03.624435 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self.session.rollback() Oct 23 19:53:03.624435 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 23 19:53:03.624435 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self.force_reraise() Oct 23 19:53:03.624435 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 23 19:53:03.624435 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise self.value Oct 23 19:53:03.624435 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 23 19:53:03.624435 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 23 19:53:03.624435 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 23 19:53:03.624435 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application session.commit() Oct 23 19:53:03.624435 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 23 19:53:03.624435 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 23 19:53:03.624435 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 23 19:53:03.625814 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self._prepare_impl() Oct 23 19:53:03.625814 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 23 19:53:03.625814 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self.session.flush() Oct 23 19:53:03.625814 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 23 19:53:03.625814 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self._flush(objects) Oct 23 19:53:03.625814 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 23 19:53:03.625814 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 23 19:53:03.625814 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 23 19:53:03.625814 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application compat.raise_( Oct 23 19:53:03.625814 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 23 19:53:03.625814 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception Oct 23 19:53:03.625814 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 23 19:53:03.625814 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application flush_context.execute() Oct 23 19:53:03.625814 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 23 19:53:03.625814 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rec.execute(self) Oct 23 19:53:03.625814 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 23 19:53:03.625814 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 23 19:53:03.626716 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 23 19:53:03.626716 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 23 19:53:03.626716 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 23 19:53:03.626716 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 23 19:53:03.626716 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 23 19:53:03.626716 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 23 19:53:03.626716 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 23 19:53:03.626716 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 23 19:53:03.626716 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 23 19:53:03.626716 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 23 19:53:03.626716 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 23 19:53:03.626716 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 23 19:53:03.626716 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 23 19:53:03.626716 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 23 19:53:03.626716 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 23 19:53:03.626716 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception Oct 23 19:53:03.627522 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 23 19:53:03.627522 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 23 19:53:03.627522 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 23 19:53:03.627522 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 23 19:53:03.627522 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 23 19:53:03.627522 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application result = self._query(query) Oct 23 19:53:03.627522 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 23 19:53:03.627522 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application conn.query(q) Oct 23 19:53:03.627522 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 23 19:53:03.627522 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 23 19:53:03.627522 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 23 19:53:03.627522 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application result.read() Oct 23 19:53:03.627522 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 23 19:53:03.627522 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 23 19:53:03.627522 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 23 19:53:03.627522 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 23 19:53:03.628379 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 23 19:53:03.628379 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 23 19:53:03.628379 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 23 19:53:03.628379 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 23 19:53:03.628379 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 23 19:53:03.628379 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 23 19:53:03.628379 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [parameters: {'id': 'e421c7eaf04f4bf99b8c550814f95194', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 23 19:53:03.628379 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 23 19:53:03.628379 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:03.628379 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 19:53:03.628379 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:03.628379 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:03.628379 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:03.628379 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:03.628379 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:03.628379 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:03.629153 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:03.629153 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:03.629153 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:03.629153 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:03.629153 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:03.629153 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:03.629153 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 23 19:53:03.629153 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 23 19:53:03.629153 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:03.629153 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:03.629153 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 23 19:53:03.629153 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 23 19:53:03.629153 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 23 19:53:03.629153 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 23 19:53:03.629153 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 23 19:53:03.629153 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:03.633214 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 177/359] 10.222.0.13 () {60 vars in 1131 bytes} [Sun Oct 23 19:53:03 2022] POST /identity/v3/roles => generated 144 bytes in 212 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:03.639129 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b547f5-1deb-43fb-aec0-5e419f7d1ef5 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:03.769828 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b547f5-1deb-43fb-aec0-5e419f7d1ef5 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:03.780325 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:03.816997 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438c901-bcc1-4f13-9372-7cb9ffc58a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:03.817983 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438c901-bcc1-4f13-9372-7cb9ffc58a57 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:03.818485 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438c901-bcc1-4f13-9372-7cb9ffc58a57 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:03.818980 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438c901-bcc1-4f13-9372-7cb9ffc58a57 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:03.835089 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8438c901-bcc1-4f13-9372-7cb9ffc58a57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:53:03.840025 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-8438c901-bcc1-4f13-9372-7cb9ffc58a57 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 19:53:03.840025 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:03.840025 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:03.840025 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:03.840025 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:03.840025 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:03.840025 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:03.840025 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:03.840025 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:03.840025 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:03.840025 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:03.840025 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:03.840025 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:53:03.840025 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:53:03.840025 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:53:03.840025 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:53:03.840931 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:53:03.840931 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:53:03.840931 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:53:03.840931 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:53:03.840931 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:03.840931 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:03.840931 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:03.840931 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:03.840931 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:03.840931 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:03.840931 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:03.840931 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:03.840931 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:03.840931 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:03.840931 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:03.840931 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:03.841739 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:03.841739 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:03.841739 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:03.841739 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:03.841739 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:53:03.841739 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:53:03.841739 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:53:03.841739 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:53:03.841739 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:53:03.841739 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:53:03.841739 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 19:53:03.841739 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:03.841739 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 183/360] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Oct 23 19:53:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:03.848802 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b2473-3646-4d1a-b079-ca13fab62a24 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:03.991305 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b2473-3646-4d1a-b079-ca13fab62a24 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:04.027161 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca95d51-4bef-455d-81ba-4b06ffbf6f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:04.027782 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca95d51-4bef-455d-81ba-4b06ffbf6f60 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:04.028066 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca95d51-4bef-455d-81ba-4b06ffbf6f60 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:04.028341 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca95d51-4bef-455d-81ba-4b06ffbf6f60 None admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:04.029022 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca95d51-4bef-455d-81ba-4b06ffbf6f60 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:04.029378 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca95d51-4bef-455d-81ba-4b06ffbf6f60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:04.030892 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca95d51-4bef-455d-81ba-4b06ffbf6f60 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:04.039929 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 178/361] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:53:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:04.655944 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba810c5-0775-4d20-983e-3d2ad78e6255 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:04.656963 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba810c5-0775-4d20-983e-3d2ad78e6255 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:04.657452 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba810c5-0775-4d20-983e-3d2ad78e6255 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:04.659354 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 184/362] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:04.665431 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05ff08-dd6e-40ae-aa6e-84f8bc58dfe3 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:04.666271 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05ff08-dd6e-40ae-aa6e-84f8bc58dfe3 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:04.666755 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05ff08-dd6e-40ae-aa6e-84f8bc58dfe3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:04.692544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-cc05ff08-dd6e-40ae-aa6e-84f8bc58dfe3 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:04.694065 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-cc05ff08-dd6e-40ae-aa6e-84f8bc58dfe3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:04.709371 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:04.790665 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 179/363] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 19:53:05.149907 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba810c5-0775-4d20-983e-3d2ad78e6255 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:05.229987 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453142b4-6f5b-459f-818d-29721747702f None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:05.230352 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453142b4-6f5b-459f-818d-29721747702f None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:05.230600 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453142b4-6f5b-459f-818d-29721747702f None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:05.232112 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 180/364] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:05.243434 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588720f-fe8b-4606-848c-fb52d214dc86 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:05.243993 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588720f-fe8b-4606-848c-fb52d214dc86 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:05.244483 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9588720f-fe8b-4606-848c-fb52d214dc86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:05.260684 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba810c5-0775-4d20-983e-3d2ad78e6255 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:05.275953 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-9588720f-fe8b-4606-848c-fb52d214dc86 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:05.277341 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-9588720f-fe8b-4606-848c-fb52d214dc86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:05.315804 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8b104-9017-419f-8651-d3d458e07d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:05.316725 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8b104-9017-419f-8651-d3d458e07d23 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:05.318810 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8b104-9017-419f-8651-d3d458e07d23 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:05.319234 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8b104-9017-419f-8651-d3d458e07d23 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:05.341373 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc8b104-9017-419f-8651-d3d458e07d23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:53:05.362420 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-5bc8b104-9017-419f-8651-d3d458e07d23 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 19:53:05.362420 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:05.362420 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:05.362420 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:05.362420 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:05.362420 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:05.362420 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:05.362420 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:05.362420 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:05.362420 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:05.362420 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:05.362420 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:05.362420 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:53:05.362420 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:53:05.362420 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:53:05.362420 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:53:05.363577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:53:05.363577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:53:05.363577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:53:05.363577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:53:05.363577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:05.363577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:05.363577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:05.363577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:05.363577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:05.363577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:05.363577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:05.363577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:05.363577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:05.363577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:05.363577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:05.363577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:05.364531 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:05.364531 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:05.364531 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:05.364531 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:05.364531 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:53:05.364531 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:53:05.364531 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:53:05.364531 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:53:05.364531 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:53:05.364531 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:53:05.364531 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 19:53:05.364531 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:05.364531 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 185/365] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Oct 23 19:53:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:05.371780 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8b104-9017-419f-8651-d3d458e07d23 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:05.396497 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 181/366] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:53:05.430667 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:05.534109 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8b104-9017-419f-8651-d3d458e07d23 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:05.579820 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e4f05-24d1-4342-98ff-9e31929d98cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:05.580439 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e4f05-24d1-4342-98ff-9e31929d98cf None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:05.580719 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e4f05-24d1-4342-98ff-9e31929d98cf None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:05.580992 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e4f05-24d1-4342-98ff-9e31929d98cf None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:05.581663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837e4f05-24d1-4342-98ff-9e31929d98cf None admin] RBAC: Adding query filter params (name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:05.582139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837e4f05-24d1-4342-98ff-9e31929d98cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:05.583565 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837e4f05-24d1-4342-98ff-9e31929d98cf None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:05.593654 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 186/367] 10.222.0.13 () {56 vars in 1103 bytes} [Sun Oct 23 19:53:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:05.599208 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9588720f-fe8b-4606-848c-fb52d214dc86 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:05.700748 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:05.736202 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9588720f-fe8b-4606-848c-fb52d214dc86 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:05.785751 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a481f6-dbad-46af-86c7-2b99995e6271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:05.786349 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a481f6-dbad-46af-86c7-2b99995e6271 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:05.786632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a481f6-dbad-46af-86c7-2b99995e6271 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:05.786904 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a481f6-dbad-46af-86c7-2b99995e6271 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:05.792866 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732007e5-e62b-4b50-8b7b-dbad8d06c2cb None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:05.793748 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732007e5-e62b-4b50-8b7b-dbad8d06c2cb None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:05.794235 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732007e5-e62b-4b50-8b7b-dbad8d06c2cb None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:05.798344 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 187/368] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 19:53:05.800045 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a481f6-dbad-46af-86c7-2b99995e6271 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:05.800351 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a481f6-dbad-46af-86c7-2b99995e6271 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:05.801789 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a57db5-f614-4b78-b231-85887896d6bb None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:05.802335 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a57db5-f614-4b78-b231-85887896d6bb None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:05.802826 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a57db5-f614-4b78-b231-85887896d6bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:05.821120 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-65a481f6-dbad-46af-86c7-2b99995e6271 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:05.821120 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:05.821120 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:05.821120 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:05.821120 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:05.821120 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:05.821120 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:05.821120 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:05.821120 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:05.821120 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:05.821120 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:05.821120 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:05.821120 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:05.821120 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:05.821120 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:05.821120 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:05.822068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:05.822068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:05.822068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:05.822068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:05.822068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:05.822068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:05.822068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:05.822068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:05.822068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:05.822068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:05.822068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:05.822068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:05.822068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:05.822068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:05.822068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:05.822068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:05.822068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:05.823366 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:05.823366 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:05.823366 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:05.823366 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 182/369] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:05.828113 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a481f6-dbad-46af-86c7-2b99995e6271 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:05.845616 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-41a57db5-f614-4b78-b231-85887896d6bb None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:05.847202 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-41a57db5-f614-4b78-b231-85887896d6bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:05.931338 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 188/370] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 19:53:05.937838 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a57db5-f614-4b78-b231-85887896d6bb None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:05.954760 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a481f6-dbad-46af-86c7-2b99995e6271 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:05.982611 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:06.013767 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:06.028340 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd92f933-11ec-4e25-847b-201aac9af3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:06.029027 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92f933-11ec-4e25-847b-201aac9af3aa None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:06.029374 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92f933-11ec-4e25-847b-201aac9af3aa None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:06.029733 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd92f933-11ec-4e25-847b-201aac9af3aa None admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:06.030539 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92f933-11ec-4e25-847b-201aac9af3aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:06.030986 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92f933-11ec-4e25-847b-201aac9af3aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:06.032419 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd92f933-11ec-4e25-847b-201aac9af3aa None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:06.045039 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 183/371] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:53:06.046304 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db6f3b-bc93-49a4-9d08-e5d5d69a5ca0 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:06.047164 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db6f3b-bc93-49a4-9d08-e5d5d69a5ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:06.047505 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db6f3b-bc93-49a4-9d08-e5d5d69a5ca0 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:06.052835 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 184/372] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:06.053687 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db6f3b-bc93-49a4-9d08-e5d5d69a5ca0 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:06.095021 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a57db5-f614-4b78-b231-85887896d6bb None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:06.131684 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18ad812-e24a-4f24-95ae-0025a3a2a017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:06.132421 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ad812-e24a-4f24-95ae-0025a3a2a017 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:06.132759 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ad812-e24a-4f24-95ae-0025a3a2a017 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:06.133041 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18ad812-e24a-4f24-95ae-0025a3a2a017 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:06.144712 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18ad812-e24a-4f24-95ae-0025a3a2a017 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:53:06.150133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-c18ad812-e24a-4f24-95ae-0025a3a2a017 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 19:53:06.150133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:06.150133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:06.150133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:06.150133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:06.150133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:06.150133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:06.150133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:06.150133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:06.150133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:06.150133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:06.150133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:06.150133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:53:06.150133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:53:06.150133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:53:06.150133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:53:06.151045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:53:06.151045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:53:06.151045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:53:06.151045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:53:06.151045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:06.151045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:06.151045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:06.151045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:06.151045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:06.151045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:06.151045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:06.151045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:06.151045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:06.151045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:06.151045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:06.151045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:06.152195 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:06.152195 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:06.152195 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:06.152195 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:06.152195 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:53:06.152195 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:53:06.152195 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:53:06.152195 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:53:06.152195 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:53:06.152195 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:53:06.152195 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 19:53:06.152195 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:06.152195 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 189/373] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Oct 23 19:53:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:06.153298 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536266e-1614-48d8-b248-711a7bd30360 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:06.153581 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536266e-1614-48d8-b248-711a7bd30360 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:06.153971 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536266e-1614-48d8-b248-711a7bd30360 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:06.157811 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db6f3b-bc93-49a4-9d08-e5d5d69a5ca0 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:06.173237 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-c536266e-1614-48d8-b248-711a7bd30360 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:06.174514 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c536266e-1614-48d8-b248-711a7bd30360 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:06.193126 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a3a4a-828f-4fd9-a364-63975b8b3b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:06.193883 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a3a4a-828f-4fd9-a364-63975b8b3b09 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:06.194274 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a3a4a-828f-4fd9-a364-63975b8b3b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:06.194651 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25a3a4a-828f-4fd9-a364-63975b8b3b09 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:06.201976 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25a3a4a-828f-4fd9-a364-63975b8b3b09 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:06.204087 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25a3a4a-828f-4fd9-a364-63975b8b3b09 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:06.217269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-c25a3a4a-828f-4fd9-a364-63975b8b3b09 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 23 19:53:06.217269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:06.217269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:06.217269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:06.217269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:06.217269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:06.217269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:06.217269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:06.217269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:06.217269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:06.217269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:06.217269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:06.217269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 19:53:06.217269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 19:53:06.217269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 19:53:06.217269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 19:53:06.219166 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:06.219166 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:06.219166 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 19:53:06.219166 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:06.219166 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 19:53:06.219166 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:06.219166 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:06.219166 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:06.219166 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:06.219166 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:06.219166 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:06.219166 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:06.219166 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:06.219166 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:06.219166 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:06.219166 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:06.219166 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:06.219166 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:06.221613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:06.221613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:06.221613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 19:53:06.221613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 19:53:06.221613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 19:53:06.221613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 19:53:06.221613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 19:53:06.221613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 19:53:06.221613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 23 19:53:06.221613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:06.221613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 185/374] 10.222.0.13 () {56 vars in 1093 bytes} [Sun Oct 23 19:53:06 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:06.221613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a3a4a-828f-4fd9-a364-63975b8b3b09 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:06.254144 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 190/375] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:53:06.255097 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c536266e-1614-48d8-b248-711a7bd30360 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:06.268537 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:06.274699 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:06.342617 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25a3a4a-828f-4fd9-a364-63975b8b3b09 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:06.373810 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c536266e-1614-48d8-b248-711a7bd30360 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:06.389608 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69adeb40-15a1-4c9d-9a43-7f47e6de6967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:06.391342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69adeb40-15a1-4c9d-9a43-7f47e6de6967 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:06.391813 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69adeb40-15a1-4c9d-9a43-7f47e6de6967 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:06.392486 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69adeb40-15a1-4c9d-9a43-7f47e6de6967 None admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:06.393755 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69adeb40-15a1-4c9d-9a43-7f47e6de6967 None admin] RBAC: Adding query filter params (name=service) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:06.394260 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69adeb40-15a1-4c9d-9a43-7f47e6de6967 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:06.395907 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69adeb40-15a1-4c9d-9a43-7f47e6de6967 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:06.408820 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 186/376] 10.222.0.13 () {56 vars in 1103 bytes} [Sun Oct 23 19:53:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:06.409937 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69adeb40-15a1-4c9d-9a43-7f47e6de6967 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:06.415582 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a817e4-2b05-4250-b457-f58a81a02cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:06.416333 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a817e4-2b05-4250-b457-f58a81a02cae None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:06.416688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a817e4-2b05-4250-b457-f58a81a02cae None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:06.417034 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a817e4-2b05-4250-b457-f58a81a02cae None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:06.417849 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a817e4-2b05-4250-b457-f58a81a02cae None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:06.418226 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a817e4-2b05-4250-b457-f58a81a02cae None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:06.419773 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a817e4-2b05-4250-b457-f58a81a02cae None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:06.464854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 191/377] 10.222.0.13 () {56 vars in 1133 bytes} [Sun Oct 23 19:53:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:06.464854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a817e4-2b05-4250-b457-f58a81a02cae None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:06.539651 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69adeb40-15a1-4c9d-9a43-7f47e6de6967 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:06.549728 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:06.552824 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:06.577264 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c4cd7-11f3-4a0a-873a-2a9962e65176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:06.577959 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c4cd7-11f3-4a0a-873a-2a9962e65176 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:06.578294 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c4cd7-11f3-4a0a-873a-2a9962e65176 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:06.578593 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8c4cd7-11f3-4a0a-873a-2a9962e65176 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:06.589035 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a817e4-2b05-4250-b457-f58a81a02cae None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:06.589447 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8c4cd7-11f3-4a0a-873a-2a9962e65176 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:53:06.595468 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-4f8c4cd7-11f3-4a0a-873a-2a9962e65176 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 19:53:06.595468 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:06.595468 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:06.595468 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:06.595468 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:06.595468 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:06.595468 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:06.595468 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:06.595468 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:06.595468 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:06.595468 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:06.595468 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:06.595468 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:53:06.595468 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:53:06.595468 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:53:06.595468 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:53:06.596381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:53:06.596381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:53:06.596381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:53:06.596381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:53:06.596381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:06.596381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:06.596381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:06.596381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:06.596381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:06.596381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:06.596381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:06.596381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:06.596381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:06.596381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:06.596381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:06.596381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:06.597192 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:06.597192 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:06.597192 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:06.597192 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:06.597192 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:53:06.597192 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:53:06.597192 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:53:06.597192 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:53:06.597192 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:53:06.597192 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:53:06.597192 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 19:53:06.597192 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:06.601283 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 187/378] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Oct 23 19:53:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:06.602169 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c4cd7-11f3-4a0a-873a-2a9962e65176 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:06.627149 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4544f9d-be8d-4c08-9461-784e593fbdbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:06.627782 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4544f9d-be8d-4c08-9461-784e593fbdbd None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:06.628067 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4544f9d-be8d-4c08-9461-784e593fbdbd None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:06.628411 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4544f9d-be8d-4c08-9461-784e593fbdbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:06.638870 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4544f9d-be8d-4c08-9461-784e593fbdbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:06.640366 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4544f9d-be8d-4c08-9461-784e593fbdbd None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:06.658097 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-e4544f9d-be8d-4c08-9461-784e593fbdbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:06.658097 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:06.658097 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:06.658097 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:06.658097 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:06.658097 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:06.658097 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:06.658097 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:06.658097 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:06.658097 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:06.658097 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:06.658097 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:06.658097 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:06.658097 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:06.658097 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:06.658097 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:06.659219 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:06.659219 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:06.659219 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:06.659219 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:06.659219 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:06.659219 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:06.659219 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:06.659219 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:06.659219 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:06.659219 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:06.659219 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:06.659219 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:06.659219 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:06.659219 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:06.659219 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:06.659219 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:06.659219 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:06.660257 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:06.660257 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:06.660257 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:06.661896 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 192/379] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:06.662841 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4544f9d-be8d-4c08-9461-784e593fbdbd None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:06.714535 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8c4cd7-11f3-4a0a-873a-2a9962e65176 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:06.752443 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184293c-6a21-4d51-83de-168539b0ad01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:06.753084 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184293c-6a21-4d51-83de-168539b0ad01 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:06.753366 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184293c-6a21-4d51-83de-168539b0ad01 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:06.753694 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f184293c-6a21-4d51-83de-168539b0ad01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:06.766878 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f184293c-6a21-4d51-83de-168539b0ad01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:06.768507 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f184293c-6a21-4d51-83de-168539b0ad01 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:06.775253 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4544f9d-be8d-4c08-9461-784e593fbdbd None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:06.783793 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-f184293c-6a21-4d51-83de-168539b0ad01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:06.783793 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:06.783793 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:06.783793 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:06.783793 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:06.783793 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:06.783793 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:06.783793 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:06.783793 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:06.783793 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:06.783793 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:06.783793 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:06.783793 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:06.783793 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:06.783793 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:06.783793 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:06.784739 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:06.784739 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:06.784739 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:06.784739 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:06.784739 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:06.784739 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:06.784739 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:06.784739 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:06.784739 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:06.784739 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:06.784739 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:06.784739 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:06.784739 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:06.784739 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:06.784739 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:06.784739 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:06.784739 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:06.790419 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:06.790419 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:06.790419 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:06.790419 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 188/380] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:06.790419 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184293c-6a21-4d51-83de-168539b0ad01 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:06.803221 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:06.813905 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:06.818818 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955af92-5e12-4020-8784-3630e08514c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:06.819449 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955af92-5e12-4020-8784-3630e08514c4 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:06.819723 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955af92-5e12-4020-8784-3630e08514c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:06.819988 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0955af92-5e12-4020-8784-3630e08514c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:06.831737 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0955af92-5e12-4020-8784-3630e08514c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:53:06.837864 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-0955af92-5e12-4020-8784-3630e08514c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 19:53:06.837864 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:06.837864 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:06.837864 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:06.837864 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:06.837864 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:06.837864 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:06.837864 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:06.837864 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:06.837864 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:06.837864 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:06.837864 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:06.837864 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:53:06.837864 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:53:06.837864 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:53:06.837864 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:53:06.838854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:53:06.838854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:53:06.838854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:53:06.838854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:53:06.838854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:06.838854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:06.838854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:06.838854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:06.838854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:06.838854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:06.838854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:06.838854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:06.838854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:06.838854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:06.838854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:06.838854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:06.839648 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:06.839648 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:06.839648 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:06.839648 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:06.839648 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:53:06.839648 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:53:06.839648 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:53:06.839648 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:53:06.839648 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:53:06.839648 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:53:06.839648 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 19:53:06.839648 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:06.839648 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 193/381] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Oct 23 19:53:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:06.840371 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955af92-5e12-4020-8784-3630e08514c4 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:06.894601 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f184293c-6a21-4d51-83de-168539b0ad01 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:06.928599 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafaafad-efc9-428c-bad3-fdb1b8283873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:06.929205 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafaafad-efc9-428c-bad3-fdb1b8283873 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:06.929490 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafaafad-efc9-428c-bad3-fdb1b8283873 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:06.929769 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafaafad-efc9-428c-bad3-fdb1b8283873 None admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:06.930471 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafaafad-efc9-428c-bad3-fdb1b8283873 None admin] RBAC: Adding query filter params (name=service) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:06.930776 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafaafad-efc9-428c-bad3-fdb1b8283873 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:06.932157 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafaafad-efc9-428c-bad3-fdb1b8283873 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:06.940271 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 189/382] 10.222.0.13 () {56 vars in 1103 bytes} [Sun Oct 23 19:53:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:06.940414 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0955af92-5e12-4020-8784-3630e08514c4 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:06.941109 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafaafad-efc9-428c-bad3-fdb1b8283873 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:06.974075 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757a1ad-87e7-4c86-8ce7-677f76a8eeb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:06.974712 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757a1ad-87e7-4c86-8ce7-677f76a8eeb7 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:06.974991 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757a1ad-87e7-4c86-8ce7-677f76a8eeb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:06.975264 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757a1ad-87e7-4c86-8ce7-677f76a8eeb7 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:06.975898 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b757a1ad-87e7-4c86-8ce7-677f76a8eeb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:06.976206 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b757a1ad-87e7-4c86-8ce7-677f76a8eeb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:06.977562 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b757a1ad-87e7-4c86-8ce7-677f76a8eeb7 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:06.989556 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 194/383] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:06.989556 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4447deb-d417-4d85-834a-ab2e6ebf3f6a None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:06.989556 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4447deb-d417-4d85-834a-ab2e6ebf3f6a None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:06.989556 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4447deb-d417-4d85-834a-ab2e6ebf3f6a None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:06.989556 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 195/384] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:06.989556 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4447deb-d417-4d85-834a-ab2e6ebf3f6a None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:07.031448 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:07.037564 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:07.050999 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafaafad-efc9-428c-bad3-fdb1b8283873 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:07.084124 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74d06c-b3fd-4f61-a023-de48f9ac7db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:07.084723 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74d06c-b3fd-4f61-a023-de48f9ac7db6 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:07.084998 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74d06c-b3fd-4f61-a023-de48f9ac7db6 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:07.085272 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74d06c-b3fd-4f61-a023-de48f9ac7db6 None admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:07.085907 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b74d06c-b3fd-4f61-a023-de48f9ac7db6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:07.086212 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b74d06c-b3fd-4f61-a023-de48f9ac7db6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:07.087663 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b74d06c-b3fd-4f61-a023-de48f9ac7db6 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:07.096259 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4447deb-d417-4d85-834a-ab2e6ebf3f6a None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:07.099121 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 190/385] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:07.099912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74d06c-b3fd-4f61-a023-de48f9ac7db6 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:07.130953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ae6c2a-266f-4510-b036-b0882a22b22b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:07.131548 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ae6c2a-266f-4510-b036-b0882a22b22b None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:07.131823 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ae6c2a-266f-4510-b036-b0882a22b22b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:07.132094 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ae6c2a-266f-4510-b036-b0882a22b22b None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:07.132772 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ae6c2a-266f-4510-b036-b0882a22b22b None admin] RBAC: Adding query filter params (name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:07.133069 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ae6c2a-266f-4510-b036-b0882a22b22b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:07.134446 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ae6c2a-266f-4510-b036-b0882a22b22b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:07.142599 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 196/386] 10.222.0.13 () {56 vars in 1103 bytes} [Sun Oct 23 19:53:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:07.143373 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ae6c2a-266f-4510-b036-b0882a22b22b None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:07.198903 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74d06c-b3fd-4f61-a023-de48f9ac7db6 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:07.231967 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e3a77b-8e7d-44f9-9440-76e47bd36660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:07.232587 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3a77b-8e7d-44f9-9440-76e47bd36660 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:07.232866 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3a77b-8e7d-44f9-9440-76e47bd36660 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:07.233392 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e3a77b-8e7d-44f9-9440-76e47bd36660 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:07.241660 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ae6c2a-266f-4510-b036-b0882a22b22b None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:07.242871 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e3a77b-8e7d-44f9-9440-76e47bd36660 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:07.243156 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:07.244464 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e3a77b-8e7d-44f9-9440-76e47bd36660 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:07.246181 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:07.260114 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-a5e3a77b-8e7d-44f9-9440-76e47bd36660 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:07.260114 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:07.260114 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:07.260114 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:07.260114 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:07.260114 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:07.260114 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:07.260114 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:07.260114 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:07.260114 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:07.260114 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:07.260114 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:07.260114 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:07.260114 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:07.260114 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:07.260114 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:07.261008 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:07.261008 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:07.261008 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:07.261008 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:07.261008 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:07.261008 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:07.261008 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:07.261008 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:07.261008 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:07.261008 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:07.261008 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:07.261008 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:07.261008 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:07.261008 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:07.261008 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:07.261008 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:07.261008 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:07.262063 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:07.262063 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:07.262063 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:07.262063 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 191/387] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:07.262431 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ec848-f07e-4e30-a5b1-9cb35947fa74 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:07.262644 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ec848-f07e-4e30-a5b1-9cb35947fa74 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:07.262890 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ec848-f07e-4e30-a5b1-9cb35947fa74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:07.279941 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-823ec848-f07e-4e30-a5b1-9cb35947fa74 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:07.280110 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287f313-01a1-4173-964e-9c41fc2dc874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:07.280653 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287f313-01a1-4173-964e-9c41fc2dc874 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:07.281438 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287f313-01a1-4173-964e-9c41fc2dc874 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:07.281665 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-823ec848-f07e-4e30-a5b1-9cb35947fa74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:07.281763 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3287f313-01a1-4173-964e-9c41fc2dc874 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:07.289087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3287f313-01a1-4173-964e-9c41fc2dc874 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:07.290528 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3287f313-01a1-4173-964e-9c41fc2dc874 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:07.302705 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-3287f313-01a1-4173-964e-9c41fc2dc874 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 23 19:53:07.302705 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:07.302705 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:07.302705 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:07.302705 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:07.302705 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:07.302705 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:07.302705 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:07.302705 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:07.302705 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:07.302705 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:07.302705 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:07.302705 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 19:53:07.302705 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 19:53:07.302705 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 19:53:07.302705 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 19:53:07.303568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:07.303568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:07.303568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 19:53:07.303568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:07.303568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 19:53:07.303568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:07.303568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:07.303568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:07.303568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:07.303568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:07.303568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:07.303568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:07.303568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:07.303568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:07.303568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:07.303568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:07.303568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:07.303568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:07.304503 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:07.304503 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:07.304503 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 19:53:07.304503 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 19:53:07.304503 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 19:53:07.304503 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 19:53:07.304503 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 19:53:07.304503 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 19:53:07.304503 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 23 19:53:07.304503 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:07.304503 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 197/388] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:53:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:07.304503 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287f313-01a1-4173-964e-9c41fc2dc874 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:07.352190 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 192/389] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:53:07.353023 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ec848-f07e-4e30-a5b1-9cb35947fa74 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:07.408279 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3287f313-01a1-4173-964e-9c41fc2dc874 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:07.443523 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810623f2-172d-4eb2-a16d-4ea8f80d91e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:07.444145 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810623f2-172d-4eb2-a16d-4ea8f80d91e8 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:07.444423 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810623f2-172d-4eb2-a16d-4ea8f80d91e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:07.444695 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810623f2-172d-4eb2-a16d-4ea8f80d91e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:07.451565 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810623f2-172d-4eb2-a16d-4ea8f80d91e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:07.453003 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810623f2-172d-4eb2-a16d-4ea8f80d91e8 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:07.455089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ec848-f07e-4e30-a5b1-9cb35947fa74 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:07.464515 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-810623f2-172d-4eb2-a16d-4ea8f80d91e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:07.464515 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:07.464515 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:07.464515 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:07.464515 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:07.464515 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:07.464515 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:07.464515 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:07.464515 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:07.464515 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:07.464515 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:07.464515 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:07.464515 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 19:53:07.464515 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 19:53:07.464515 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 19:53:07.465377 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 19:53:07.465377 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:07.465377 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:07.465377 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:07.465377 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:07.465377 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:07.465377 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:07.465377 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:07.465377 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:07.465377 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:07.465377 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:07.465377 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:07.465377 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:07.465377 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:07.465377 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:07.465377 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:07.465377 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:07.466274 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 19:53:07.466274 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 19:53:07.466274 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 19:53:07.466274 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 19:53:07.466274 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 19:53:07.466274 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 19:53:07.466274 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:07.466274 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:07.466274 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 198/390] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:53:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:07.466817 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810623f2-172d-4eb2-a16d-4ea8f80d91e8 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:07.490937 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:07.493776 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:07.506846 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75d8b5-9538-403c-a853-4f7ad2662237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:07.507481 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75d8b5-9538-403c-a853-4f7ad2662237 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:07.507757 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75d8b5-9538-403c-a853-4f7ad2662237 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:07.508031 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c75d8b5-9538-403c-a853-4f7ad2662237 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:07.520876 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c75d8b5-9538-403c-a853-4f7ad2662237 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:07.522350 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c75d8b5-9538-403c-a853-4f7ad2662237 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:07.541782 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-3c75d8b5-9538-403c-a853-4f7ad2662237 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:07.541782 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:07.541782 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:07.541782 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:07.541782 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:07.541782 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:07.541782 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:07.541782 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:07.541782 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:07.541782 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:07.541782 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:07.541782 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:07.541782 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:07.541782 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:07.541782 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:07.541782 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:07.542985 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:07.542985 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:07.542985 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:07.542985 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:07.542985 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:07.542985 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:07.542985 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:07.542985 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:07.542985 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:07.542985 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:07.542985 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:07.542985 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:07.542985 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:07.542985 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:07.542985 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:07.542985 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:07.542985 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:07.543824 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:07.543824 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:07.543824 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:07.544476 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 193/391] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:07.545358 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75d8b5-9538-403c-a853-4f7ad2662237 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:07.573819 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810623f2-172d-4eb2-a16d-4ea8f80d91e8 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:07.611315 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b8241-2a8c-4954-b7a6-0c5fba7bef63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:07.611927 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b8241-2a8c-4954-b7a6-0c5fba7bef63 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:07.612197 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b8241-2a8c-4954-b7a6-0c5fba7bef63 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:07.612467 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3b8241-2a8c-4954-b7a6-0c5fba7bef63 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:07.613105 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b8241-2a8c-4954-b7a6-0c5fba7bef63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:07.613761 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b8241-2a8c-4954-b7a6-0c5fba7bef63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:07.614805 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3b8241-2a8c-4954-b7a6-0c5fba7bef63 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:07.627796 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 199/392] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:07.628671 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b8241-2a8c-4954-b7a6-0c5fba7bef63 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:07.648808 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c75d8b5-9538-403c-a853-4f7ad2662237 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:07.681597 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a43b8b-5a62-4f9a-a748-dbf116c98f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:07.682193 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a43b8b-5a62-4f9a-a748-dbf116c98f3c None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:07.682498 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a43b8b-5a62-4f9a-a748-dbf116c98f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:07.682772 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a43b8b-5a62-4f9a-a748-dbf116c98f3c None admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:07.683493 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a43b8b-5a62-4f9a-a748-dbf116c98f3c None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:07.683797 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a43b8b-5a62-4f9a-a748-dbf116c98f3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:07.685203 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a43b8b-5a62-4f9a-a748-dbf116c98f3c None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:07.712578 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:07.721783 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 194/393] 10.222.0.13 () {56 vars in 1137 bytes} [Sun Oct 23 19:53:07 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:07.722884 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a43b8b-5a62-4f9a-a748-dbf116c98f3c None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:07.731906 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3b8241-2a8c-4954-b7a6-0c5fba7bef63 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:07.758732 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:07.767000 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f7820-e720-44be-96a7-16ed297e6d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:07.767713 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f7820-e720-44be-96a7-16ed297e6d82 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:07.767993 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f7820-e720-44be-96a7-16ed297e6d82 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:07.768274 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618f7820-e720-44be-96a7-16ed297e6d82 None admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:07.768965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618f7820-e720-44be-96a7-16ed297e6d82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:07.769363 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618f7820-e720-44be-96a7-16ed297e6d82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:07.770788 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618f7820-e720-44be-96a7-16ed297e6d82 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:07.788992 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 200/394] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Oct 23 19:53:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:07.789777 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f7820-e720-44be-96a7-16ed297e6d82 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:07.824063 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a43b8b-5a62-4f9a-a748-dbf116c98f3c None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:07.857111 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b05f7-6338-4dde-a000-0435dcd68332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:07.857781 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b05f7-6338-4dde-a000-0435dcd68332 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:07.858061 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b05f7-6338-4dde-a000-0435dcd68332 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:07.858359 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9b05f7-6338-4dde-a000-0435dcd68332 None admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:07.859496 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b05f7-6338-4dde-a000-0435dcd68332 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:07.859792 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b05f7-6338-4dde-a000-0435dcd68332 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:07.861186 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9b05f7-6338-4dde-a000-0435dcd68332 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:07.871996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 195/395] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:07.872785 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b05f7-6338-4dde-a000-0435dcd68332 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:07.891128 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618f7820-e720-44be-96a7-16ed297e6d82 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:07.923601 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446eb0d3-98e9-4d49-a4ff-52a6fd1aebb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:07.924281 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446eb0d3-98e9-4d49-a4ff-52a6fd1aebb3 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:07.924560 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446eb0d3-98e9-4d49-a4ff-52a6fd1aebb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:07.924857 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446eb0d3-98e9-4d49-a4ff-52a6fd1aebb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:07.934686 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446eb0d3-98e9-4d49-a4ff-52a6fd1aebb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:07.936330 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446eb0d3-98e9-4d49-a4ff-52a6fd1aebb3 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:07.939230 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:07.953608 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-446eb0d3-98e9-4d49-a4ff-52a6fd1aebb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:07.953608 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:07.953608 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:07.953608 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:07.953608 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:07.953608 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:07.953608 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:07.953608 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:07.953608 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:07.953608 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:07.953608 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:07.953608 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:07.953608 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:07.953608 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:07.953608 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:07.953608 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:07.954609 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:07.954609 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:07.954609 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:07.954609 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:07.954609 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:07.954609 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:07.954609 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:07.954609 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:07.954609 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:07.954609 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:07.954609 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:07.954609 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:07.954609 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:07.954609 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:07.954609 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:07.954609 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:07.954609 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:07.955509 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:07.955509 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:07.955509 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:07.955509 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 201/396] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:07.955509 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446eb0d3-98e9-4d49-a4ff-52a6fd1aebb3 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:07.965848 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:07.973296 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9b05f7-6338-4dde-a000-0435dcd68332 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:08.005798 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13bb071-f5b2-4594-ba3a-9593e7902aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:08.006481 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13bb071-f5b2-4594-ba3a-9593e7902aff None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:08.006761 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13bb071-f5b2-4594-ba3a-9593e7902aff None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:08.007036 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13bb071-f5b2-4594-ba3a-9593e7902aff None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:08.014479 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13bb071-f5b2-4594-ba3a-9593e7902aff None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:08.016097 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13bb071-f5b2-4594-ba3a-9593e7902aff None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:08.032052 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-c13bb071-f5b2-4594-ba3a-9593e7902aff None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 23 19:53:08.032052 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:08.032052 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:08.032052 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:08.032052 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:08.032052 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:08.032052 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:08.032052 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:08.032052 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:08.032052 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:08.032052 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:08.032052 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:08.032052 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 19:53:08.032052 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 19:53:08.032052 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 19:53:08.032052 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 19:53:08.033019 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:08.033019 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:08.033019 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 19:53:08.033019 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:08.033019 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 19:53:08.033019 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:08.033019 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:08.033019 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:08.033019 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:08.033019 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:08.033019 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:08.033019 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:08.033019 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:08.033019 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:08.033019 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:08.033019 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:08.033019 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:08.033019 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:08.033896 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:08.033896 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:08.033896 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 19:53:08.033896 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 19:53:08.033896 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 19:53:08.033896 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 19:53:08.033896 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 19:53:08.033896 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 19:53:08.033896 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 23 19:53:08.033896 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:08.033896 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 196/397] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:53:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:08.033896 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13bb071-f5b2-4594-ba3a-9593e7902aff None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:08.057770 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446eb0d3-98e9-4d49-a4ff-52a6fd1aebb3 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:08.090731 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3e3705-2244-4ee2-90c4-74bd17f5c6b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:08.091354 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e3705-2244-4ee2-90c4-74bd17f5c6b6 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:08.091632 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e3705-2244-4ee2-90c4-74bd17f5c6b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:08.091931 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3e3705-2244-4ee2-90c4-74bd17f5c6b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:08.102724 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3e3705-2244-4ee2-90c4-74bd17f5c6b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:08.104172 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3e3705-2244-4ee2-90c4-74bd17f5c6b6 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:08.118764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-5d3e3705-2244-4ee2-90c4-74bd17f5c6b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:08.118764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:08.118764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:08.118764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:08.118764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:08.118764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:08.118764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:08.118764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:08.118764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:08.118764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:08.118764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:08.118764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:08.118764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:08.118764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:08.118764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:08.118764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:08.119674 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:08.119674 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:08.119674 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:08.119674 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:08.119674 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:08.119674 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:08.119674 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:08.119674 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:08.119674 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:08.119674 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:08.119674 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:08.119674 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:08.119674 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:08.119674 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:08.119674 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:08.119674 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:08.119674 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:08.120514 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:08.120514 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:08.120514 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:08.120514 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 202/398] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:08.120735 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3e3705-2244-4ee2-90c4-74bd17f5c6b6 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:08.136618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13bb071-f5b2-4594-ba3a-9593e7902aff None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:08.165774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:08.169982 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5624be1c-a456-4837-99ad-951d7b2a786d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:08.170749 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624be1c-a456-4837-99ad-951d7b2a786d None admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:08.171028 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624be1c-a456-4837-99ad-951d7b2a786d None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:08.171300 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5624be1c-a456-4837-99ad-951d7b2a786d None admin] PATH_INFO: `/v3/projects/d40947427f5f43e7b6c82e3605737767/users/0b3d512304254ffcaa8dad90abf1f9c1/roles/5e0524ede3ee479c923c7b609a0ca402` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:08.185745 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5624be1c-a456-4837-99ad-951d7b2a786d None admin] RBAC: Authorizing `identity:create_grant(project_id=d40947427f5f43e7b6c82e3605737767, user_id=0b3d512304254ffcaa8dad90abf1f9c1, role_id=5e0524ede3ee479c923c7b609a0ca402)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:08.187265 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5624be1c-a456-4837-99ad-951d7b2a786d None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:08.211141 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 197/399] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Oct 23 19:53:08 2022] PUT /identity/v3/projects/d40947427f5f43e7b6c82e3605737767/users/0b3d512304254ffcaa8dad90abf1f9c1/roles/5e0524ede3ee479c923c7b609a0ca402 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 19:53:08.211141 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5624be1c-a456-4837-99ad-951d7b2a786d None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:08.211141 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:08.235582 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3e3705-2244-4ee2-90c4-74bd17f5c6b6 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:08.279430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b92c2bb-4d45-47a6-bff0-adc2f89cc878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:08.280171 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92c2bb-4d45-47a6-bff0-adc2f89cc878 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:08.280449 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92c2bb-4d45-47a6-bff0-adc2f89cc878 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:08.280749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92c2bb-4d45-47a6-bff0-adc2f89cc878 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:08.290149 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b92c2bb-4d45-47a6-bff0-adc2f89cc878 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:08.291749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b92c2bb-4d45-47a6-bff0-adc2f89cc878 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:08.306807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-5b92c2bb-4d45-47a6-bff0-adc2f89cc878 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 23 19:53:08.306807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:08.306807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:08.306807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:08.306807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:08.306807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:08.306807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:08.306807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:08.306807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:08.306807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:08.306807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:08.306807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:08.306807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 19:53:08.306807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 19:53:08.306807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 19:53:08.306807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 19:53:08.307774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:08.307774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:08.307774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 19:53:08.307774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:08.307774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 19:53:08.307774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:08.307774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:08.307774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:08.307774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:08.307774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:08.307774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:08.307774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:08.307774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:08.307774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:08.307774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:08.307774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:08.307774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:08.307774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:08.308680 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:08.308680 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:08.308680 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 19:53:08.308680 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 19:53:08.308680 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 19:53:08.308680 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 19:53:08.308680 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 19:53:08.308680 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 19:53:08.308680 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 23 19:53:08.308680 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:08.308680 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 203/400] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Oct 23 19:53:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:08.309269 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b92c2bb-4d45-47a6-bff0-adc2f89cc878 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:08.328206 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5624be1c-a456-4837-99ad-951d7b2a786d None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:08.366931 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551348fe-4d5d-48d7-bbb9-a591eea77ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:08.367549 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551348fe-4d5d-48d7-bbb9-a591eea77ec2 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:08.367832 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551348fe-4d5d-48d7-bbb9-a591eea77ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:08.368108 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551348fe-4d5d-48d7-bbb9-a591eea77ec2 None admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:08.368766 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551348fe-4d5d-48d7-bbb9-a591eea77ec2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:08.369066 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551348fe-4d5d-48d7-bbb9-a591eea77ec2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:08.370466 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551348fe-4d5d-48d7-bbb9-a591eea77ec2 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:08.383923 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 198/401] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:08.384809 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551348fe-4d5d-48d7-bbb9-a591eea77ec2 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:08.423301 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b92c2bb-4d45-47a6-bff0-adc2f89cc878 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:08.431272 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:08.459670 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:08.460529 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8346db-c1a1-40c2-b5b2-d773c08ed0bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:08.461153 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8346db-c1a1-40c2-b5b2-d773c08ed0bd None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:08.461422 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8346db-c1a1-40c2-b5b2-d773c08ed0bd None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:08.461691 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8346db-c1a1-40c2-b5b2-d773c08ed0bd None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:08.462456 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8346db-c1a1-40c2-b5b2-d773c08ed0bd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:08.462772 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8346db-c1a1-40c2-b5b2-d773c08ed0bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:08.464188 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8346db-c1a1-40c2-b5b2-d773c08ed0bd None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:08.483464 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 204/402] 10.222.0.13 () {56 vars in 1137 bytes} [Sun Oct 23 19:53:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:08.484638 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8346db-c1a1-40c2-b5b2-d773c08ed0bd None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:08.504438 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551348fe-4d5d-48d7-bbb9-a591eea77ec2 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:08.538989 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7878881-785f-41a0-98c8-ffad1e9d3bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:08.539599 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7878881-785f-41a0-98c8-ffad1e9d3bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:08.539889 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7878881-785f-41a0-98c8-ffad1e9d3bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:08.540156 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7878881-785f-41a0-98c8-ffad1e9d3bd1 None admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:08.540801 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7878881-785f-41a0-98c8-ffad1e9d3bd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:08.541104 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7878881-785f-41a0-98c8-ffad1e9d3bd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:08.542498 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7878881-785f-41a0-98c8-ffad1e9d3bd1 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:08.555744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 199/403] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:08.556345 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7878881-785f-41a0-98c8-ffad1e9d3bd1 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:08.595037 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8346db-c1a1-40c2-b5b2-d773c08ed0bd None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:08.630940 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd813524-16b4-4db4-b0c8-205b36258b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:08.631555 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd813524-16b4-4db4-b0c8-205b36258b43 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:08.631833 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd813524-16b4-4db4-b0c8-205b36258b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:08.632210 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd813524-16b4-4db4-b0c8-205b36258b43 None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:08.632829 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd813524-16b4-4db4-b0c8-205b36258b43 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:08.633139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd813524-16b4-4db4-b0c8-205b36258b43 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:08.634587 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd813524-16b4-4db4-b0c8-205b36258b43 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:08.654118 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 205/404] 10.222.0.13 () {56 vars in 1139 bytes} [Sun Oct 23 19:53:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:08.655025 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd813524-16b4-4db4-b0c8-205b36258b43 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:08.669139 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7878881-785f-41a0-98c8-ffad1e9d3bd1 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:08.684092 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:08.695990 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:08.708410 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a61f05-c2b0-4f7a-8733-99d5e40b9283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:08.709153 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a61f05-c2b0-4f7a-8733-99d5e40b9283 None admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:08.709412 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a61f05-c2b0-4f7a-8733-99d5e40b9283 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:08.709683 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a61f05-c2b0-4f7a-8733-99d5e40b9283 None admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:08.710393 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a61f05-c2b0-4f7a-8733-99d5e40b9283 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:08.712170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a61f05-c2b0-4f7a-8733-99d5e40b9283 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:08.746281 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 200/405] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Oct 23 19:53:08 2022] POST /identity/v3/users => generated 247 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:53:08.746281 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a61f05-c2b0-4f7a-8733-99d5e40b9283 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:08.789049 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd813524-16b4-4db4-b0c8-205b36258b43 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:08.841506 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f67a39-2460-4d7a-a9b7-cdbc1d4f12db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:08.842271 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f67a39-2460-4d7a-a9b7-cdbc1d4f12db None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:08.843801 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f67a39-2460-4d7a-a9b7-cdbc1d4f12db None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:08.844154 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f67a39-2460-4d7a-a9b7-cdbc1d4f12db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:08.860322 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f67a39-2460-4d7a-a9b7-cdbc1d4f12db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:08.862095 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f67a39-2460-4d7a-a9b7-cdbc1d4f12db None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:08.872129 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a61f05-c2b0-4f7a-8733-99d5e40b9283 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:08.883404 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-44f67a39-2460-4d7a-a9b7-cdbc1d4f12db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:08.883404 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:08.883404 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:08.883404 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:08.883404 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:08.883404 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:08.883404 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:08.883404 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:08.883404 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:08.883404 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:08.883404 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:08.883404 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:08.883404 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:08.883404 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:08.883404 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:08.883404 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:08.884549 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:08.884549 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:08.884549 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:08.884549 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:08.884549 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:08.884549 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:08.884549 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:08.884549 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:08.884549 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:08.884549 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:08.884549 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:08.884549 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:08.884549 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:08.884549 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:08.884549 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:08.884549 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:08.884549 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:08.885599 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:08.885599 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:08.885599 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:08.886854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 206/406] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:08.888149 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f67a39-2460-4d7a-a9b7-cdbc1d4f12db None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:08.914235 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f05799-0f84-4f3b-b9ac-14c6ec9c9e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:08.916139 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f05799-0f84-4f3b-b9ac-14c6ec9c9e2c None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:08.916139 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f05799-0f84-4f3b-b9ac-14c6ec9c9e2c None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:08.916139 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f05799-0f84-4f3b-b9ac-14c6ec9c9e2c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:08.925576 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f05799-0f84-4f3b-b9ac-14c6ec9c9e2c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:08.927176 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f05799-0f84-4f3b-b9ac-14c6ec9c9e2c None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:08.941541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-92f05799-0f84-4f3b-b9ac-14c6ec9c9e2c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:08.941541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:08.941541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:08.941541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:08.941541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:08.941541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:08.941541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:08.941541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:08.941541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:08.941541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:08.941541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:08.941541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:08.941541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 19:53:08.941541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 19:53:08.941541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 19:53:08.942482 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 19:53:08.942482 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:08.942482 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:08.942482 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:08.942482 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:08.942482 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:08.942482 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:08.942482 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:08.942482 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:08.942482 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:08.942482 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:08.942482 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:08.942482 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:08.942482 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:08.942482 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:08.942482 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:08.942482 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:08.943324 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 19:53:08.943324 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 19:53:08.943324 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 19:53:08.943324 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 19:53:08.943324 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 19:53:08.943324 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 19:53:08.943324 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:08.943324 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:08.943324 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 201/407] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:53:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:08.943782 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f05799-0f84-4f3b-b9ac-14c6ec9c9e2c None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:08.987849 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:09.004043 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:09.014635 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f67a39-2460-4d7a-a9b7-cdbc1d4f12db None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:09.052461 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8bdf96-378a-4e20-b1e0-fb25a6d368cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:09.053018 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8bdf96-378a-4e20-b1e0-fb25a6d368cb None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:09.053290 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8bdf96-378a-4e20-b1e0-fb25a6d368cb None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:09.053565 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8bdf96-378a-4e20-b1e0-fb25a6d368cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:09.065152 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8bdf96-378a-4e20-b1e0-fb25a6d368cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:09.066651 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8bdf96-378a-4e20-b1e0-fb25a6d368cb None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:09.081317 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f05799-0f84-4f3b-b9ac-14c6ec9c9e2c None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:09.082139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-6a8bdf96-378a-4e20-b1e0-fb25a6d368cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:09.082139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:09.082139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:09.082139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:09.082139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:09.082139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:09.082139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:09.082139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:09.082139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:09.082139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:09.082139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:09.082139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:09.082139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:09.082139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:09.082139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:09.082139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:09.083033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:09.083033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:09.083033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:09.083033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:09.083033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:09.083033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:09.083033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:09.083033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:09.083033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:09.083033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:09.083033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:09.083033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:09.083033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:09.083033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:09.083033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:09.083033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:09.083033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:09.083954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:09.083954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:09.083954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:09.083954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 207/408] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:09.084583 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8bdf96-378a-4e20-b1e0-fb25a6d368cb None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:09.125969 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a21b686-f252-4c74-8a1d-edbac4002c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:09.126882 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a21b686-f252-4c74-8a1d-edbac4002c4a None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:09.127169 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a21b686-f252-4c74-8a1d-edbac4002c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:09.127451 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a21b686-f252-4c74-8a1d-edbac4002c4a None admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:09.128748 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a21b686-f252-4c74-8a1d-edbac4002c4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:09.129077 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a21b686-f252-4c74-8a1d-edbac4002c4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:09.130761 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a21b686-f252-4c74-8a1d-edbac4002c4a None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:09.147319 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 202/409] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:09.148163 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a21b686-f252-4c74-8a1d-edbac4002c4a None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:09.210975 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8bdf96-378a-4e20-b1e0-fb25a6d368cb None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:09.238010 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:09.248984 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc141188-b3c3-4975-bd2e-2b23afbaa3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:09.249816 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc141188-b3c3-4975-bd2e-2b23afbaa3aa None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:09.250222 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc141188-b3c3-4975-bd2e-2b23afbaa3aa None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:09.250623 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc141188-b3c3-4975-bd2e-2b23afbaa3aa None admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:09.251412 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc141188-b3c3-4975-bd2e-2b23afbaa3aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:09.251814 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc141188-b3c3-4975-bd2e-2b23afbaa3aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:09.253344 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc141188-b3c3-4975-bd2e-2b23afbaa3aa None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:09.268095 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 208/410] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Oct 23 19:53:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:09.269465 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc141188-b3c3-4975-bd2e-2b23afbaa3aa None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:09.277305 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:09.281631 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a21b686-f252-4c74-8a1d-edbac4002c4a None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:09.322396 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa9548-37ce-4719-a12d-33ac538b117c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:09.323130 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa9548-37ce-4719-a12d-33ac538b117c None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:09.323412 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa9548-37ce-4719-a12d-33ac538b117c None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:09.323688 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa9548-37ce-4719-a12d-33ac538b117c None admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:09.324334 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa9548-37ce-4719-a12d-33ac538b117c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:09.324639 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa9548-37ce-4719-a12d-33ac538b117c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:09.326156 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa9548-37ce-4719-a12d-33ac538b117c None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:09.341370 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 203/411] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:09.345433 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa9548-37ce-4719-a12d-33ac538b117c None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:09.399612 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc141188-b3c3-4975-bd2e-2b23afbaa3aa None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:09.442766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb08e4d0-300e-4a41-a5c9-21095be1446f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:09.443406 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb08e4d0-300e-4a41-a5c9-21095be1446f None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:09.443687 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb08e4d0-300e-4a41-a5c9-21095be1446f None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:09.443987 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb08e4d0-300e-4a41-a5c9-21095be1446f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:09.452669 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb08e4d0-300e-4a41-a5c9-21095be1446f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:09.454176 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa9548-37ce-4719-a12d-33ac538b117c None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:09.454448 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb08e4d0-300e-4a41-a5c9-21095be1446f None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:09.466386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-fb08e4d0-300e-4a41-a5c9-21095be1446f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:09.466386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:09.466386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:09.466386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:09.466386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:09.466386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:09.466386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:09.466386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:09.466386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:09.466386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:09.466386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:09.466386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:09.466386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 19:53:09.466386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 19:53:09.466386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 19:53:09.467477 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 19:53:09.467477 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:09.467477 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:09.467477 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:09.467477 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:09.467477 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:09.467477 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:09.467477 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:09.467477 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:09.467477 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:09.467477 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:09.467477 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:09.467477 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:09.467477 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:09.467477 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:09.467477 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:09.467477 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:09.468859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 19:53:09.468859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 19:53:09.468859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 19:53:09.468859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 19:53:09.468859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 19:53:09.468859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 19:53:09.468859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:09.468859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:09.468859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 209/412] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:53:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:09.468859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb08e4d0-300e-4a41-a5c9-21095be1446f None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:09.496307 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5b04a-7f57-4537-9221-6bbcbe73941a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:09.497218 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5b04a-7f57-4537-9221-6bbcbe73941a None admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:09.497665 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5b04a-7f57-4537-9221-6bbcbe73941a None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:09.498085 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd5b04a-7f57-4537-9221-6bbcbe73941a None admin] PATH_INFO: `/v3/projects/d40947427f5f43e7b6c82e3605737767/users/021216f90d7f4ff6b6eb7a1d86399d99/roles/5e0524ede3ee479c923c7b609a0ca402` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:09.514413 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd5b04a-7f57-4537-9221-6bbcbe73941a None admin] RBAC: Authorizing `identity:create_grant(project_id=d40947427f5f43e7b6c82e3605737767, user_id=021216f90d7f4ff6b6eb7a1d86399d99, role_id=5e0524ede3ee479c923c7b609a0ca402)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:09.516301 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd5b04a-7f57-4537-9221-6bbcbe73941a None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:09.527317 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:09.535324 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 204/413] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Oct 23 19:53:09 2022] PUT /identity/v3/projects/d40947427f5f43e7b6c82e3605737767/users/021216f90d7f4ff6b6eb7a1d86399d99/roles/5e0524ede3ee479c923c7b609a0ca402 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 19:53:09.536978 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5b04a-7f57-4537-9221-6bbcbe73941a None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:09.555697 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:09.596558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb08e4d0-300e-4a41-a5c9-21095be1446f None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:09.636554 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f368f49-a468-4d21-9a9a-67546baa2ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:09.637199 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f368f49-a468-4d21-9a9a-67546baa2ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:09.637481 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f368f49-a468-4d21-9a9a-67546baa2ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:09.637758 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f368f49-a468-4d21-9a9a-67546baa2ca3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:09.646687 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f368f49-a468-4d21-9a9a-67546baa2ca3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:09.648185 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f368f49-a468-4d21-9a9a-67546baa2ca3 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:09.674688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-8f368f49-a468-4d21-9a9a-67546baa2ca3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:09.674688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:09.674688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:09.674688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:09.674688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:09.674688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:09.674688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:09.674688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:09.674688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:09.674688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:09.674688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:09.674688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:09.674688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 19:53:09.674688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 19:53:09.674688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 19:53:09.675754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 19:53:09.675754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:09.675754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:09.675754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:09.675754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:09.675754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:09.675754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:09.675754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:09.675754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:09.675754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:09.675754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:09.675754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:09.675754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:09.675754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:09.675754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:09.675754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:09.675754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:09.676644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 19:53:09.676644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 19:53:09.676644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 19:53:09.676644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 19:53:09.676644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 19:53:09.676644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 19:53:09.676644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:09.676644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:09.676644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 210/414] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:53:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:09.684558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f368f49-a468-4d21-9a9a-67546baa2ca3 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:09.711010 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd5b04a-7f57-4537-9221-6bbcbe73941a None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:09.783371 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2313be39-9017-4b15-a09e-340133714843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:09.784010 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313be39-9017-4b15-a09e-340133714843 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:09.784240 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313be39-9017-4b15-a09e-340133714843 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:09.784517 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2313be39-9017-4b15-a09e-340133714843 None admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:09.785218 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2313be39-9017-4b15-a09e-340133714843 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:09.785527 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2313be39-9017-4b15-a09e-340133714843 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:09.787100 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2313be39-9017-4b15-a09e-340133714843 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:09.806878 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 205/415] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Oct 23 19:53:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:09.816359 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2313be39-9017-4b15-a09e-340133714843 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:09.829095 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f368f49-a468-4d21-9a9a-67546baa2ca3 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:09.837211 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:09.873010 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c004c63-cfcf-4f7e-b110-e682401c14f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:09.873574 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c004c63-cfcf-4f7e-b110-e682401c14f9 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:09.873857 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c004c63-cfcf-4f7e-b110-e682401c14f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:09.874129 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c004c63-cfcf-4f7e-b110-e682401c14f9 None admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:09.875037 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c004c63-cfcf-4f7e-b110-e682401c14f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:09.875351 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c004c63-cfcf-4f7e-b110-e682401c14f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:09.876781 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c004c63-cfcf-4f7e-b110-e682401c14f9 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:09.890572 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 211/416] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Oct 23 19:53:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:09.898889 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c004c63-cfcf-4f7e-b110-e682401c14f9 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:09.917287 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:09.982959 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2313be39-9017-4b15-a09e-340133714843 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:10.054239 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aac1d1-6971-49c2-942e-fe6311c434ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:10.055790 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aac1d1-6971-49c2-942e-fe6311c434ac None admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:10.056070 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aac1d1-6971-49c2-942e-fe6311c434ac None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:10.057560 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aac1d1-6971-49c2-942e-fe6311c434ac None admin] PATH_INFO: `/v3/projects/d40947427f5f43e7b6c82e3605737767/users/7bcc5f5101fb4913aaf5caf564ca6b60/roles/5e0524ede3ee479c923c7b609a0ca402` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:10.088168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aac1d1-6971-49c2-942e-fe6311c434ac None admin] RBAC: Authorizing `identity:create_grant(project_id=d40947427f5f43e7b6c82e3605737767, user_id=7bcc5f5101fb4913aaf5caf564ca6b60, role_id=5e0524ede3ee479c923c7b609a0ca402)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:10.089624 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aac1d1-6971-49c2-942e-fe6311c434ac None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:10.099091 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c004c63-cfcf-4f7e-b110-e682401c14f9 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:10.115229 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 206/417] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Oct 23 19:53:09 2022] PUT /identity/v3/projects/d40947427f5f43e7b6c82e3605737767/users/7bcc5f5101fb4913aaf5caf564ca6b60/roles/5e0524ede3ee479c923c7b609a0ca402 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 19:53:10.152114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54351f-5543-4e01-9f2d-060f01091f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:10.153044 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54351f-5543-4e01-9f2d-060f01091f19 None admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:10.153479 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54351f-5543-4e01-9f2d-060f01091f19 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:10.153897 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54351f-5543-4e01-9f2d-060f01091f19 None admin] PATH_INFO: `/v3/projects/d40947427f5f43e7b6c82e3605737767/users/ba8ff3413bc549d29486d1c724a1f383/roles/5e0524ede3ee479c923c7b609a0ca402` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:10.180362 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54351f-5543-4e01-9f2d-060f01091f19 None admin] RBAC: Authorizing `identity:create_grant(project_id=d40947427f5f43e7b6c82e3605737767, user_id=ba8ff3413bc549d29486d1c724a1f383, role_id=5e0524ede3ee479c923c7b609a0ca402)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:10.182114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54351f-5543-4e01-9f2d-060f01091f19 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:10.209533 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 212/418] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Oct 23 19:53:09 2022] PUT /identity/v3/projects/d40947427f5f43e7b6c82e3605737767/users/ba8ff3413bc549d29486d1c724a1f383/roles/5e0524ede3ee479c923c7b609a0ca402 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 19:53:10.555598 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455f3f1-9773-4a68-8ccf-cc3ae2c14bd1 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:10.556047 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455f3f1-9773-4a68-8ccf-cc3ae2c14bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:10.556434 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455f3f1-9773-4a68-8ccf-cc3ae2c14bd1 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:10.558159 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 207/419] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:10.563773 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af382f1-a15b-4b54-b8de-d7759059f2f7 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:10.564171 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af382f1-a15b-4b54-b8de-d7759059f2f7 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:10.564547 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af382f1-a15b-4b54-b8de-d7759059f2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:10.583449 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-7af382f1-a15b-4b54-b8de-d7759059f2f7 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:10.584784 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7af382f1-a15b-4b54-b8de-d7759059f2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:10.596367 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:10.656821 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 213/420] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:53:11.081438 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455f3f1-9773-4a68-8ccf-cc3ae2c14bd1 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:11.227935 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455f3f1-9773-4a68-8ccf-cc3ae2c14bd1 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:11.251302 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:11.268676 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec61cd4-3bc0-4949-823a-e6c5c29e0f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:11.269081 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6c3d7-11ea-4a1b-a166-a640da8ac405 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:11.269512 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6c3d7-11ea-4a1b-a166-a640da8ac405 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:11.269669 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6c3d7-11ea-4a1b-a166-a640da8ac405 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:11.270520 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec61cd4-3bc0-4949-823a-e6c5c29e0f57 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:11.270998 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec61cd4-3bc0-4949-823a-e6c5c29e0f57 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:11.271248 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 214/421] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:11.271680 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec61cd4-3bc0-4949-823a-e6c5c29e0f57 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:11.275978 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c0e4b-0e6c-416c-a7fd-3980cec0d66c None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:11.276495 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c0e4b-0e6c-416c-a7fd-3980cec0d66c None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:11.276977 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271c0e4b-0e6c-416c-a7fd-3980cec0d66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:11.298914 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec61cd4-3bc0-4949-823a-e6c5c29e0f57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:53:11.307093 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-aec61cd4-3bc0-4949-823a-e6c5c29e0f57 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 19:53:11.307093 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:11.307093 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:11.307093 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:11.307093 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:11.307093 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:11.307093 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:11.307093 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:11.307093 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:11.307093 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:11.307093 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:11.307093 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:11.307093 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:53:11.307093 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:53:11.307093 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:53:11.307093 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:53:11.307996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:53:11.307996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:53:11.307996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:53:11.307996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:53:11.307996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:11.307996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:11.307996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:11.307996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:11.307996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:11.307996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:11.307996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:11.307996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:11.307996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:11.307996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:11.307996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:11.307996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:11.308837 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:11.308837 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:11.308837 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:11.308837 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:11.308837 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:53:11.308837 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:53:11.308837 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:53:11.308837 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:53:11.308837 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:53:11.308837 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:53:11.308837 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 19:53:11.308837 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:11.310959 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-271c0e4b-0e6c-416c-a7fd-3980cec0d66c None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:11.312413 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-271c0e4b-0e6c-416c-a7fd-3980cec0d66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:11.314020 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 208/422] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Oct 23 19:53:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:11.319968 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec61cd4-3bc0-4949-823a-e6c5c29e0f57 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:11.431854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 215/423] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 19:53:11.473640 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec61cd4-3bc0-4949-823a-e6c5c29e0f57 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:11.525551 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271c470-ea95-4d0d-bc29-181ed809b3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:11.526514 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271c470-ea95-4d0d-bc29-181ed809b3b3 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:11.527050 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271c470-ea95-4d0d-bc29-181ed809b3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:11.527522 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b271c470-ea95-4d0d-bc29-181ed809b3b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:11.528463 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b271c470-ea95-4d0d-bc29-181ed809b3b3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:11.528961 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b271c470-ea95-4d0d-bc29-181ed809b3b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:11.530577 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b271c470-ea95-4d0d-bc29-181ed809b3b3 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:11.539673 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 209/424] 10.222.0.13 () {56 vars in 1103 bytes} [Sun Oct 23 19:53:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:53:11.547806 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c0e4b-0e6c-416c-a7fd-3980cec0d66c None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:11.648267 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:11.675294 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271c0e4b-0e6c-416c-a7fd-3980cec0d66c None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:11.716656 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5fc194-0a88-49b5-98dc-c800c1044229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:11.717292 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5fc194-0a88-49b5-98dc-c800c1044229 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:11.717571 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5fc194-0a88-49b5-98dc-c800c1044229 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:11.717845 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5fc194-0a88-49b5-98dc-c800c1044229 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:11.736908 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5fc194-0a88-49b5-98dc-c800c1044229 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:11.738374 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5fc194-0a88-49b5-98dc-c800c1044229 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:11.756360 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-ab5fc194-0a88-49b5-98dc-c800c1044229 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:11.756360 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:11.756360 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:11.756360 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:11.756360 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:11.756360 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:11.756360 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:11.756360 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:11.756360 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:11.756360 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:11.756360 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:11.756360 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:11.756360 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:11.756360 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:11.756360 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:11.756360 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:11.757254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:11.757254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:11.757254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:11.757254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:11.757254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:11.757254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:11.757254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:11.757254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:11.757254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:11.757254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:11.757254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:11.757254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:11.757254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:11.757254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:11.757254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:11.757254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:11.757254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:11.758229 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:11.758229 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:11.758229 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:11.758229 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 216/425] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:11.763152 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271c470-ea95-4d0d-bc29-181ed809b3b3 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:11.833663 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:11.915691 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b271c470-ea95-4d0d-bc29-181ed809b3b3 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:11.940121 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5fc194-0a88-49b5-98dc-c800c1044229 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:11.966447 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08625807-5f8c-416e-9f27-20bb1a1ac6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:11.967835 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08625807-5f8c-416e-9f27-20bb1a1ac6c7 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:11.968645 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08625807-5f8c-416e-9f27-20bb1a1ac6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:11.968930 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08625807-5f8c-416e-9f27-20bb1a1ac6c7 None admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:11.969583 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08625807-5f8c-416e-9f27-20bb1a1ac6c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:11.969888 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08625807-5f8c-416e-9f27-20bb1a1ac6c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:11.971284 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08625807-5f8c-416e-9f27-20bb1a1ac6c7 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:11.994022 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 210/426] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:12.000815 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08625807-5f8c-416e-9f27-20bb1a1ac6c7 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:12.117693 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5fc194-0a88-49b5-98dc-c800c1044229 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:12.172367 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:12.189662 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f967879d-fa0a-441c-a706-c60dc141906d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:12.191448 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967879d-fa0a-441c-a706-c60dc141906d None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:12.192111 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967879d-fa0a-441c-a706-c60dc141906d None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:12.194368 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f967879d-fa0a-441c-a706-c60dc141906d None admin] PATH_INFO: `/v3/roles/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:12.201950 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08625807-5f8c-416e-9f27-20bb1a1ac6c7 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:12.204640 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:12.220576 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f967879d-fa0a-441c-a706-c60dc141906d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:53:12.228766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-f967879d-fa0a-441c-a706-c60dc141906d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 19:53:12.228766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:12.228766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:12.228766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:12.228766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:12.228766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:12.228766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:12.228766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:12.228766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:12.228766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:12.228766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:12.228766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:12.228766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:53:12.228766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:53:12.228766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:53:12.228766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:53:12.229650 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:53:12.229650 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:53:12.229650 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:53:12.229650 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:53:12.229650 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:12.229650 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:12.229650 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:12.229650 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:12.229650 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:12.229650 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:12.229650 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:12.229650 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:12.229650 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:12.229650 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:12.229650 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:12.229650 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:12.230475 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:12.230475 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:12.230475 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:12.230475 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:12.230475 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:53:12.230475 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:53:12.230475 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:53:12.230475 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:53:12.230475 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:53:12.230475 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:53:12.230475 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 19:53:12.230475 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:12.233539 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 217/427] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Oct 23 19:53:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:12.240061 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f967879d-fa0a-441c-a706-c60dc141906d None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:12.274432 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d5c2b-fb0f-4f1e-a72b-ba81573fdcda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:12.275075 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d5c2b-fb0f-4f1e-a72b-ba81573fdcda None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:12.275363 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d5c2b-fb0f-4f1e-a72b-ba81573fdcda None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:12.275635 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d5c2b-fb0f-4f1e-a72b-ba81573fdcda None admin] PATH_INFO: `/v3/users/nova` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:12.295411 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564d5c2b-fb0f-4f1e-a72b-ba81573fdcda None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:12.297419 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564d5c2b-fb0f-4f1e-a72b-ba81573fdcda None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:12.323529 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-564d5c2b-fb0f-4f1e-a72b-ba81573fdcda None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 23 19:53:12.323529 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:12.323529 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:12.323529 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:12.323529 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:12.323529 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:12.323529 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:12.323529 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:12.323529 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:12.323529 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:12.323529 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:12.323529 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:12.323529 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 19:53:12.323529 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 19:53:12.323529 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 19:53:12.323529 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 19:53:12.324489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:12.324489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:12.324489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 19:53:12.324489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:12.324489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 19:53:12.324489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:12.324489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:12.324489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:12.324489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:12.324489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:12.324489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:12.324489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:12.324489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:12.324489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:12.324489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:12.324489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:12.324489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:12.324489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:12.325369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:12.325369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:12.325369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 19:53:12.325369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 19:53:12.325369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 19:53:12.325369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 19:53:12.325369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 19:53:12.325369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 19:53:12.325369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 23 19:53:12.325369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:12.325369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 211/428] 10.222.0.13 () {56 vars in 1093 bytes} [Sun Oct 23 19:53:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:12.330698 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d5c2b-fb0f-4f1e-a72b-ba81573fdcda None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:12.361706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f967879d-fa0a-441c-a706-c60dc141906d None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:12.398615 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e0411-661c-4f09-850c-25e22720bd20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:12.399805 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e0411-661c-4f09-850c-25e22720bd20 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:12.400306 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e0411-661c-4f09-850c-25e22720bd20 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:12.400798 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572e0411-661c-4f09-850c-25e22720bd20 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:12.401661 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572e0411-661c-4f09-850c-25e22720bd20 None admin] RBAC: Adding query filter params (name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:12.402163 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572e0411-661c-4f09-850c-25e22720bd20 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:12.404199 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572e0411-661c-4f09-850c-25e22720bd20 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:12.414062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 218/429] 10.222.0.13 () {56 vars in 1103 bytes} [Sun Oct 23 19:53:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:53:12.419443 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e0411-661c-4f09-850c-25e22720bd20 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:12.495208 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:12.503899 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d5c2b-fb0f-4f1e-a72b-ba81573fdcda None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:12.540581 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9397a005-dd60-4e25-9bf0-9a51f4e61071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:12.541569 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397a005-dd60-4e25-9bf0-9a51f4e61071 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:12.542086 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397a005-dd60-4e25-9bf0-9a51f4e61071 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:12.542585 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397a005-dd60-4e25-9bf0-9a51f4e61071 None admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:12.543519 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9397a005-dd60-4e25-9bf0-9a51f4e61071 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:12.544027 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9397a005-dd60-4e25-9bf0-9a51f4e61071 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:12.545664 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9397a005-dd60-4e25-9bf0-9a51f4e61071 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:12.552522 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572e0411-661c-4f09-850c-25e22720bd20 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:12.566603 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 212/430] 10.222.0.13 () {56 vars in 1133 bytes} [Sun Oct 23 19:53:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:12.567851 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa8c54-b084-4fec-80d5-e201babb40fd None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:12.568120 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa8c54-b084-4fec-80d5-e201babb40fd None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:12.568392 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa8c54-b084-4fec-80d5-e201babb40fd None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:12.571035 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 213/431] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:12.571035 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeabb68-676f-4bce-8186-5c4470edc8b5 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:12.571211 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeabb68-676f-4bce-8186-5c4470edc8b5 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:12.571400 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeabb68-676f-4bce-8186-5c4470edc8b5 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:12.572700 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 214/432] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:12.574400 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeabb68-676f-4bce-8186-5c4470edc8b5 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:12.598145 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a539fd-f435-4f28-beaa-7eef7ad80427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:12.598371 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:12.598811 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a539fd-f435-4f28-beaa-7eef7ad80427 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:12.599092 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a539fd-f435-4f28-beaa-7eef7ad80427 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:12.599384 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a539fd-f435-4f28-beaa-7eef7ad80427 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:12.610656 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a539fd-f435-4f28-beaa-7eef7ad80427 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:12.612093 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a539fd-f435-4f28-beaa-7eef7ad80427 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:12.627933 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-71a539fd-f435-4f28-beaa-7eef7ad80427 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:12.627933 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:12.627933 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:12.627933 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:12.627933 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:12.627933 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:12.627933 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:12.627933 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:12.627933 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:12.627933 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:12.627933 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:12.627933 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:12.627933 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:12.627933 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:12.627933 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:12.627933 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:12.628859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:12.628859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:12.628859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:12.628859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:12.628859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:12.628859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:12.628859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:12.628859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:12.628859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:12.628859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:12.628859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:12.628859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:12.628859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:12.628859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:12.628859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:12.628859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:12.628859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:12.629698 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:12.629698 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:12.629698 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:12.633720 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 219/433] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:12.634943 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0be28b-20e1-4c9b-915a-39f146a7ef23 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:12.635225 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0be28b-20e1-4c9b-915a-39f146a7ef23 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:12.635499 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0be28b-20e1-4c9b-915a-39f146a7ef23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:12.653911 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-cb0be28b-20e1-4c9b-915a-39f146a7ef23 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:12.655048 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-cb0be28b-20e1-4c9b-915a-39f146a7ef23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:12.685511 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeabb68-676f-4bce-8186-5c4470edc8b5 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:12.722311 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077aeb92-2bf4-47b1-b095-875c19a3d712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:12.723194 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077aeb92-2bf4-47b1-b095-875c19a3d712 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:12.723689 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077aeb92-2bf4-47b1-b095-875c19a3d712 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:12.724132 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077aeb92-2bf4-47b1-b095-875c19a3d712 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:12.725747 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 220/434] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:53:12.727274 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36794cc6-12c3-4db1-833b-2c969bae51cd None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:12.727562 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36794cc6-12c3-4db1-833b-2c969bae51cd None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:12.727836 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36794cc6-12c3-4db1-833b-2c969bae51cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:12.759507 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-36794cc6-12c3-4db1-833b-2c969bae51cd None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:12.760962 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-36794cc6-12c3-4db1-833b-2c969bae51cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:12.796984 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:12.800002 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077aeb92-2bf4-47b1-b095-875c19a3d712 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:12.801644 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077aeb92-2bf4-47b1-b095-875c19a3d712 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:12.824386 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:12.862704 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 221/435] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:53:12.864463 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36794cc6-12c3-4db1-833b-2c969bae51cd None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:12.954680 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 215/436] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Oct 23 19:53:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 19:53:12.958035 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5434c-ea88-4b24-a211-c722f356c980 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:12.958345 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5434c-ea88-4b24-a211-c722f356c980 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:12.958618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f5434c-ea88-4b24-a211-c722f356c980 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:12.959915 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 216/437] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:12.960603 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5434c-ea88-4b24-a211-c722f356c980 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:13.025888 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36794cc6-12c3-4db1-833b-2c969bae51cd None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:13.062430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb07d5ca-78e2-420f-854f-ac12876554bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:13.063034 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07d5ca-78e2-420f-854f-ac12876554bc None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:13.063316 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07d5ca-78e2-420f-854f-ac12876554bc None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:13.063841 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07d5ca-78e2-420f-854f-ac12876554bc None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:13.064480 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb07d5ca-78e2-420f-854f-ac12876554bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:13.064775 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb07d5ca-78e2-420f-854f-ac12876554bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:13.066114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb07d5ca-78e2-420f-854f-ac12876554bc None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:13.077450 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 222/438] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:13.078084 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f5434c-ea88-4b24-a211-c722f356c980 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:13.083373 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853b36b-e364-4871-91bf-e6a5bcf18130 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:13.083659 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853b36b-e364-4871-91bf-e6a5bcf18130 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:13.083934 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f853b36b-e364-4871-91bf-e6a5bcf18130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:13.099509 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:13.101187 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-f853b36b-e364-4871-91bf-e6a5bcf18130 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:13.102467 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f853b36b-e364-4871-91bf-e6a5bcf18130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:13.115514 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c086840-cd9a-4c31-990a-9415f43bdfcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:13.116240 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c086840-cd9a-4c31-990a-9415f43bdfcc None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:13.116528 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c086840-cd9a-4c31-990a-9415f43bdfcc None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:13.116803 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c086840-cd9a-4c31-990a-9415f43bdfcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:13.120204 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:13.128145 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c086840-cd9a-4c31-990a-9415f43bdfcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:13.129597 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c086840-cd9a-4c31-990a-9415f43bdfcc None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:13.146129 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-3c086840-cd9a-4c31-990a-9415f43bdfcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:13.146129 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:13.146129 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:13.146129 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:13.146129 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:13.146129 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:13.146129 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:13.146129 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:13.146129 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:13.146129 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:13.146129 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:13.146129 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:13.146129 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:13.146129 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:13.146129 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:13.146129 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:13.147032 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:13.147032 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:13.147032 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:13.147032 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:13.147032 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:13.147032 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:13.147032 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:13.147032 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:13.147032 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:13.147032 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:13.147032 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:13.147032 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:13.147032 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:13.147032 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:13.147032 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:13.147032 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:13.147032 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:13.147908 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:13.147908 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:13.147908 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:13.148982 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 217/439] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:13.149790 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c086840-cd9a-4c31-990a-9415f43bdfcc None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:13.175728 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 223/440] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:53:13.176771 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f853b36b-e364-4871-91bf-e6a5bcf18130 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:13.254818 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c086840-cd9a-4c31-990a-9415f43bdfcc None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:13.282184 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f853b36b-e364-4871-91bf-e6a5bcf18130 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:13.289019 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1dceeb-5866-479a-8f16-81f8bd30a86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:13.289643 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1dceeb-5866-479a-8f16-81f8bd30a86e None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:13.289920 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1dceeb-5866-479a-8f16-81f8bd30a86e None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:13.290188 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1dceeb-5866-479a-8f16-81f8bd30a86e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:13.300380 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1dceeb-5866-479a-8f16-81f8bd30a86e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:53:13.305647 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-fc1dceeb-5866-479a-8f16-81f8bd30a86e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 19:53:13.305647 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:13.305647 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:13.305647 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:13.305647 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:13.305647 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:13.305647 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:13.305647 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:13.305647 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:13.305647 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:13.305647 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:13.305647 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:13.305647 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:53:13.305647 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:53:13.305647 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:53:13.305647 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:53:13.306541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:53:13.306541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:53:13.306541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:53:13.306541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:53:13.306541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:13.306541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:13.306541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:13.306541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:13.306541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:13.306541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:13.306541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:13.306541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:13.306541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:13.306541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:13.306541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:13.306541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:13.307873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:13.307873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:13.307873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:13.307873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:13.307873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:53:13.307873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:53:13.307873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:53:13.307873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:53:13.307873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:53:13.307873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:53:13.307873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 19:53:13.307873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:13.307873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 218/441] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Oct 23 19:53:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:13.308559 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1dceeb-5866-479a-8f16-81f8bd30a86e None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:13.325804 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d018c-b702-413c-b82f-0bb5bb9f7dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:13.326465 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d018c-b702-413c-b82f-0bb5bb9f7dca None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:13.326743 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d018c-b702-413c-b82f-0bb5bb9f7dca None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:13.327012 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d018c-b702-413c-b82f-0bb5bb9f7dca None admin] PATH_INFO: `/v3/users/swift` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:13.334693 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d018c-b702-413c-b82f-0bb5bb9f7dca None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:13.336142 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d018c-b702-413c-b82f-0bb5bb9f7dca None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:13.340491 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:13.348319 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-3c5d018c-b702-413c-b82f-0bb5bb9f7dca None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 23 19:53:13.348319 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:13.348319 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:13.348319 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:13.348319 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:13.348319 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:13.348319 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:13.348319 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:13.348319 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:13.348319 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:13.348319 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:13.348319 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:13.348319 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 19:53:13.348319 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 19:53:13.348319 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 19:53:13.348319 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 19:53:13.349295 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:13.349295 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:13.349295 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 19:53:13.349295 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:13.349295 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 19:53:13.349295 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:13.349295 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:13.349295 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:13.349295 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:13.349295 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:13.349295 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:13.349295 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:13.349295 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:13.349295 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:13.349295 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:13.349295 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:13.349295 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:13.349295 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:13.350825 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:13.350825 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:13.350825 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 19:53:13.350825 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 19:53:13.350825 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 19:53:13.350825 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 19:53:13.350825 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 19:53:13.350825 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 19:53:13.350825 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 23 19:53:13.350825 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:13.350825 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 224/442] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Oct 23 19:53:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:13.350825 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d018c-b702-413c-b82f-0bb5bb9f7dca None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:13.370226 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:13.418062 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1dceeb-5866-479a-8f16-81f8bd30a86e None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:13.453361 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8352a12d-ae60-4d8d-b84c-de603378cc01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:13.453956 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352a12d-ae60-4d8d-b84c-de603378cc01 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:13.454232 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352a12d-ae60-4d8d-b84c-de603378cc01 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:13.454536 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8352a12d-ae60-4d8d-b84c-de603378cc01 None admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:13.455169 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8352a12d-ae60-4d8d-b84c-de603378cc01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:13.455466 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8352a12d-ae60-4d8d-b84c-de603378cc01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:13.456845 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8352a12d-ae60-4d8d-b84c-de603378cc01 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:13.458489 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d018c-b702-413c-b82f-0bb5bb9f7dca None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:13.468734 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 219/443] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:13.469529 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8352a12d-ae60-4d8d-b84c-de603378cc01 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:13.491787 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eaf9ad-5e0a-484d-b970-de45a9dc1971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:13.492409 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eaf9ad-5e0a-484d-b970-de45a9dc1971 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:13.492685 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eaf9ad-5e0a-484d-b970-de45a9dc1971 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:13.492953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eaf9ad-5e0a-484d-b970-de45a9dc1971 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:13.503589 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eaf9ad-5e0a-484d-b970-de45a9dc1971 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:53:13.508908 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-17eaf9ad-5e0a-484d-b970-de45a9dc1971 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 19:53:13.508908 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:13.508908 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:13.508908 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:13.508908 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:13.508908 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:13.508908 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:13.508908 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:13.508908 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:13.508908 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:13.508908 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:13.508908 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:13.508908 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:53:13.508908 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:53:13.508908 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:53:13.508908 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:53:13.509814 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:53:13.509814 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:53:13.509814 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:53:13.509814 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:53:13.509814 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:13.509814 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:13.509814 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:13.509814 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:13.509814 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:13.509814 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:13.509814 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:13.509814 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:13.509814 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:13.509814 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:13.509814 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:13.509814 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:13.510882 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:13.510882 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:13.510882 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:13.510882 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:13.510882 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:53:13.510882 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:53:13.510882 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:53:13.510882 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:53:13.510882 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:53:13.510882 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:53:13.510882 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 19:53:13.510882 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:13.510882 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 225/444] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Oct 23 19:53:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:13.510882 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eaf9ad-5e0a-484d-b970-de45a9dc1971 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:13.561998 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:13.571428 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8352a12d-ae60-4d8d-b84c-de603378cc01 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:13.578568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:13.604540 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53309b2d-5fca-4124-944c-eeaa8c181fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:13.605144 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53309b2d-5fca-4124-944c-eeaa8c181fa7 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:13.605423 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53309b2d-5fca-4124-944c-eeaa8c181fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:13.605694 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53309b2d-5fca-4124-944c-eeaa8c181fa7 None admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:13.606429 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53309b2d-5fca-4124-944c-eeaa8c181fa7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:13.606730 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53309b2d-5fca-4124-944c-eeaa8c181fa7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:13.608080 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53309b2d-5fca-4124-944c-eeaa8c181fa7 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:13.608386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eaf9ad-5e0a-484d-b970-de45a9dc1971 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:13.616436 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 220/445] 10.222.0.13 () {56 vars in 1103 bytes} [Sun Oct 23 19:53:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:13.617203 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53309b2d-5fca-4124-944c-eeaa8c181fa7 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:13.641918 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21022f-5523-429f-b86f-7e00f4bc5d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:13.642556 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21022f-5523-429f-b86f-7e00f4bc5d0c None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:13.642838 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21022f-5523-429f-b86f-7e00f4bc5d0c None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:13.643113 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e21022f-5523-429f-b86f-7e00f4bc5d0c None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:13.643836 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21022f-5523-429f-b86f-7e00f4bc5d0c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:13.644170 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21022f-5523-429f-b86f-7e00f4bc5d0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:13.645584 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e21022f-5523-429f-b86f-7e00f4bc5d0c None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:13.662898 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 226/446] 10.222.0.13 () {56 vars in 1135 bytes} [Sun Oct 23 19:53:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:13.663690 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21022f-5523-429f-b86f-7e00f4bc5d0c None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:13.716096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53309b2d-5fca-4124-944c-eeaa8c181fa7 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:13.749297 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7496f4-64b1-4a7b-9536-4c6ce9896fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:13.749955 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7496f4-64b1-4a7b-9536-4c6ce9896fa0 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:13.750232 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7496f4-64b1-4a7b-9536-4c6ce9896fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:13.750542 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7496f4-64b1-4a7b-9536-4c6ce9896fa0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:13.760646 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7496f4-64b1-4a7b-9536-4c6ce9896fa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:53:13.761857 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e21022f-5523-429f-b86f-7e00f4bc5d0c None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:13.766350 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-9e7496f4-64b1-4a7b-9536-4c6ce9896fa0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 19:53:13.766350 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:13.766350 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:13.766350 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:13.766350 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:13.766350 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:13.766350 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:13.766350 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:13.766350 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:13.766350 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:13.766350 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:13.766350 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:13.766350 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:53:13.766350 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:53:13.766350 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:53:13.766350 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:53:13.767222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:53:13.767222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:53:13.767222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:53:13.767222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:53:13.767222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:13.767222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:13.767222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:13.767222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:13.767222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:13.767222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:13.767222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:13.767222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:13.767222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:13.767222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:13.767222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:13.767222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:13.768213 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:13.768213 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:13.768213 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:13.768213 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:13.768213 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:53:13.768213 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:53:13.768213 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:53:13.768213 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:53:13.768213 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:53:13.768213 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:53:13.768213 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 19:53:13.768213 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:13.768213 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 221/447] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Oct 23 19:53:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:13.769087 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7496f4-64b1-4a7b-9536-4c6ce9896fa0 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:13.784631 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:13.792107 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:13.798693 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ed70d-c91d-4c5b-852f-41b9dcf301cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:13.799325 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ed70d-c91d-4c5b-852f-41b9dcf301cf None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:13.799605 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ed70d-c91d-4c5b-852f-41b9dcf301cf None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:13.799877 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7ed70d-c91d-4c5b-852f-41b9dcf301cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:13.806517 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ed70d-c91d-4c5b-852f-41b9dcf301cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:13.807943 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7ed70d-c91d-4c5b-852f-41b9dcf301cf None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:13.819157 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-bc7ed70d-c91d-4c5b-852f-41b9dcf301cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:13.819157 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:13.819157 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:13.819157 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:13.819157 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:13.819157 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:13.819157 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:13.819157 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:13.819157 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:13.819157 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:13.819157 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:13.819157 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:13.819157 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 19:53:13.819157 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 19:53:13.819157 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 19:53:13.820137 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 19:53:13.820137 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:13.820137 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:13.820137 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:13.820137 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:13.820137 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:13.820137 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:13.820137 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:13.820137 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:13.820137 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:13.820137 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:13.820137 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:13.820137 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:13.820137 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:13.820137 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:13.820137 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:13.820137 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:13.820966 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 19:53:13.820966 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 19:53:13.820966 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 19:53:13.820966 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 19:53:13.820966 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 19:53:13.820966 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 19:53:13.820966 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:13.820966 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:13.820966 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 227/448] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:53:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:13.820966 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ed70d-c91d-4c5b-852f-41b9dcf301cf None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:13.872444 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7496f4-64b1-4a7b-9536-4c6ce9896fa0 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:13.904675 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dece7ff-c8e3-4752-bf3f-b1b208d19744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:13.905318 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dece7ff-c8e3-4752-bf3f-b1b208d19744 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:13.906118 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dece7ff-c8e3-4752-bf3f-b1b208d19744 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:13.906449 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dece7ff-c8e3-4752-bf3f-b1b208d19744 None admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:13.907110 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dece7ff-c8e3-4752-bf3f-b1b208d19744 None admin] RBAC: Adding query filter params (name=service) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:13.907411 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dece7ff-c8e3-4752-bf3f-b1b208d19744 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:13.908776 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dece7ff-c8e3-4752-bf3f-b1b208d19744 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:13.916423 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 222/449] 10.222.0.13 () {56 vars in 1103 bytes} [Sun Oct 23 19:53:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:13.917192 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dece7ff-c8e3-4752-bf3f-b1b208d19744 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:13.925290 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7ed70d-c91d-4c5b-852f-41b9dcf301cf None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:13.958720 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e9629a-2955-40ab-81ed-cd9d8f932705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:13.959365 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9629a-2955-40ab-81ed-cd9d8f932705 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:13.959640 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9629a-2955-40ab-81ed-cd9d8f932705 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:13.959908 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e9629a-2955-40ab-81ed-cd9d8f932705 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:13.970745 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e9629a-2955-40ab-81ed-cd9d8f932705 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:13.972160 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e9629a-2955-40ab-81ed-cd9d8f932705 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:13.985847 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-30e9629a-2955-40ab-81ed-cd9d8f932705 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:13.985847 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:13.985847 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:13.985847 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:13.985847 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:13.985847 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:13.985847 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:13.985847 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:13.985847 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:13.985847 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:13.985847 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:13.985847 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:13.985847 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:13.985847 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:13.985847 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:13.985847 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:13.986927 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:13.986927 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:13.986927 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:13.986927 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:13.986927 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:13.986927 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:13.986927 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:13.986927 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:13.986927 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:13.986927 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:13.986927 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:13.986927 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:13.986927 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:13.986927 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:13.986927 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:13.986927 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:13.986927 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:13.987763 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:13.987763 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:13.987763 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:13.987763 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 228/450] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:13.988159 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e9629a-2955-40ab-81ed-cd9d8f932705 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:13.992206 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:14.013570 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dece7ff-c8e3-4752-bf3f-b1b208d19744 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:14.034196 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:14.048348 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947039e7-aca8-461c-89f2-f2937687064c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:14.048732 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947039e7-aca8-461c-89f2-f2937687064c None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:14.049010 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947039e7-aca8-461c-89f2-f2937687064c None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:14.049279 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947039e7-aca8-461c-89f2-f2937687064c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:14.059779 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947039e7-aca8-461c-89f2-f2937687064c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:14.061223 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947039e7-aca8-461c-89f2-f2937687064c None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:14.075123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-947039e7-aca8-461c-89f2-f2937687064c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:14.075123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:14.075123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:14.075123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:14.075123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:14.075123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:14.075123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:14.075123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:14.075123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:14.075123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:14.075123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:14.075123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:14.075123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:14.075123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:14.075123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:14.075123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:14.076081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:14.076081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:14.076081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:14.076081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:14.076081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:14.076081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:14.076081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:14.076081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:14.076081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:14.076081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:14.076081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:14.076081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:14.076081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:14.076081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:14.076081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:14.076081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:14.076081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:14.076924 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:14.076924 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:14.076924 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:14.076924 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 223/451] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:14.077136 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947039e7-aca8-461c-89f2-f2937687064c None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:14.092862 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e9629a-2955-40ab-81ed-cd9d8f932705 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:14.126698 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a5d9f6-ab7e-49d0-82bb-2b3ba5c54dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:14.127303 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5d9f6-ab7e-49d0-82bb-2b3ba5c54dec None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:14.127584 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5d9f6-ab7e-49d0-82bb-2b3ba5c54dec None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:14.127855 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a5d9f6-ab7e-49d0-82bb-2b3ba5c54dec None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:14.128511 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a5d9f6-ab7e-49d0-82bb-2b3ba5c54dec None admin] RBAC: Adding query filter params (name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:14.129016 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a5d9f6-ab7e-49d0-82bb-2b3ba5c54dec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:14.130159 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a5d9f6-ab7e-49d0-82bb-2b3ba5c54dec None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:14.137821 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 229/452] 10.222.0.13 () {56 vars in 1103 bytes} [Sun Oct 23 19:53:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:14.138621 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a5d9f6-ab7e-49d0-82bb-2b3ba5c54dec None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:14.181280 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947039e7-aca8-461c-89f2-f2937687064c None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:14.207545 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:14.216134 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9287a32-ed66-4649-9a85-e2fa4973e9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:14.216734 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9287a32-ed66-4649-9a85-e2fa4973e9f2 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:14.217019 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9287a32-ed66-4649-9a85-e2fa4973e9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:14.217293 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9287a32-ed66-4649-9a85-e2fa4973e9f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:14.218013 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9287a32-ed66-4649-9a85-e2fa4973e9f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:14.218349 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9287a32-ed66-4649-9a85-e2fa4973e9f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:14.219772 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9287a32-ed66-4649-9a85-e2fa4973e9f2 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:14.231441 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 224/453] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Oct 23 19:53:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:14.232256 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9287a32-ed66-4649-9a85-e2fa4973e9f2 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:14.235554 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a5d9f6-ab7e-49d0-82bb-2b3ba5c54dec None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:14.237882 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:14.270115 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f52275-c8ba-4f54-ac0b-a9eeaf45e6d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:14.270696 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f52275-c8ba-4f54-ac0b-a9eeaf45e6d0 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:14.270978 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f52275-c8ba-4f54-ac0b-a9eeaf45e6d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:14.271249 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f52275-c8ba-4f54-ac0b-a9eeaf45e6d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:14.281501 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f52275-c8ba-4f54-ac0b-a9eeaf45e6d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:14.282988 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f52275-c8ba-4f54-ac0b-a9eeaf45e6d0 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:14.297246 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-76f52275-c8ba-4f54-ac0b-a9eeaf45e6d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:14.297246 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:14.297246 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:14.297246 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:14.297246 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:14.297246 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:14.297246 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:14.297246 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:14.297246 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:14.297246 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:14.297246 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:14.297246 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:14.297246 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:14.297246 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:14.297246 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:14.297246 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:14.298167 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:14.298167 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:14.298167 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:14.298167 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:14.298167 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:14.298167 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:14.298167 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:14.298167 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:14.298167 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:14.298167 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:14.298167 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:14.298167 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:14.298167 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:14.298167 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:14.298167 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:14.298167 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:14.298167 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:14.299072 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:14.299072 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:14.299072 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:14.299072 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 230/454] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:14.299297 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f52275-c8ba-4f54-ac0b-a9eeaf45e6d0 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:14.332187 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9287a32-ed66-4649-9a85-e2fa4973e9f2 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:14.365730 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8befe68-0bd7-4247-b9c7-ea354fcd27ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:14.366361 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8befe68-0bd7-4247-b9c7-ea354fcd27ab None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:14.366640 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8befe68-0bd7-4247-b9c7-ea354fcd27ab None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:14.366907 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8befe68-0bd7-4247-b9c7-ea354fcd27ab None admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:14.367535 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8befe68-0bd7-4247-b9c7-ea354fcd27ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:14.367833 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8befe68-0bd7-4247-b9c7-ea354fcd27ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:14.369176 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8befe68-0bd7-4247-b9c7-ea354fcd27ab None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:14.379972 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 225/455] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:14.380738 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8befe68-0bd7-4247-b9c7-ea354fcd27ab None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:14.402625 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f52275-c8ba-4f54-ac0b-a9eeaf45e6d0 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:14.428857 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:14.436819 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9fda9-9a26-4de2-bf84-63735454f6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:14.437980 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9fda9-9a26-4de2-bf84-63735454f6ea None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:14.438280 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9fda9-9a26-4de2-bf84-63735454f6ea None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:14.438811 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d9fda9-9a26-4de2-bf84-63735454f6ea None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:14.439450 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d9fda9-9a26-4de2-bf84-63735454f6ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:14.439756 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d9fda9-9a26-4de2-bf84-63735454f6ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:14.441118 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d9fda9-9a26-4de2-bf84-63735454f6ea None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:14.453309 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 231/456] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:14.454088 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9fda9-9a26-4de2-bf84-63735454f6ea None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:14.472731 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:14.480461 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8befe68-0bd7-4247-b9c7-ea354fcd27ab None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:14.513936 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb1316-c7a8-4151-8bcd-1bfe18574776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:14.514586 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb1316-c7a8-4151-8bcd-1bfe18574776 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:14.514867 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb1316-c7a8-4151-8bcd-1bfe18574776 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:14.515140 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cb1316-c7a8-4151-8bcd-1bfe18574776 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:14.525701 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cb1316-c7a8-4151-8bcd-1bfe18574776 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:14.527123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cb1316-c7a8-4151-8bcd-1bfe18574776 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:14.540961 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-e0cb1316-c7a8-4151-8bcd-1bfe18574776 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:14.540961 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:14.540961 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:14.540961 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:14.540961 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:14.540961 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:14.540961 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:14.540961 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:14.540961 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:14.540961 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:14.540961 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:14.540961 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:14.540961 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:14.540961 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:14.540961 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:14.540961 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:14.541966 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:14.541966 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:14.541966 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:14.541966 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:14.541966 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:14.541966 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:14.541966 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:14.541966 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:14.541966 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:14.541966 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:14.541966 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:14.541966 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:14.541966 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:14.541966 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:14.541966 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:14.541966 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:14.541966 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:14.543276 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:14.543276 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:14.543276 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:14.543276 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 226/457] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:14.543276 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb1316-c7a8-4151-8bcd-1bfe18574776 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:14.559072 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d9fda9-9a26-4de2-bf84-63735454f6ea None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:14.593701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062a6d5-a1eb-4622-a115-40375bfc920b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:14.594327 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062a6d5-a1eb-4622-a115-40375bfc920b None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:14.594609 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062a6d5-a1eb-4622-a115-40375bfc920b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:14.594881 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9062a6d5-a1eb-4622-a115-40375bfc920b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:14.595607 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9062a6d5-a1eb-4622-a115-40375bfc920b None admin] RBAC: Adding query filter params (role.id=5e0524ede3ee479c923c7b609a0ca402, scope.project.id=d40947427f5f43e7b6c82e3605737767, user.id=0b3d512304254ffcaa8dad90abf1f9c1) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:14.595911 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9062a6d5-a1eb-4622-a115-40375bfc920b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:14.597349 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9062a6d5-a1eb-4622-a115-40375bfc920b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:14.603088 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 232/458] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Oct 23 19:53:14 2022] GET /identity/v3/role_assignments?user.id=0b3d512304254ffcaa8dad90abf1f9c1&scope.project.id=d40947427f5f43e7b6c82e3605737767&role.id=5e0524ede3ee479c923c7b609a0ca402 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:14.604501 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062a6d5-a1eb-4622-a115-40375bfc920b None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:14.656974 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:14.660772 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cb1316-c7a8-4151-8bcd-1bfe18574776 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:14.700797 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:14.701299 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9354ef4-60ca-4bf6-89ac-ffaf94e3e294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:14.701926 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9354ef4-60ca-4bf6-89ac-ffaf94e3e294 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:14.702880 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9354ef4-60ca-4bf6-89ac-ffaf94e3e294 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:14.702880 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9354ef4-60ca-4bf6-89ac-ffaf94e3e294 None admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:14.703229 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9354ef4-60ca-4bf6-89ac-ffaf94e3e294 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:14.703837 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9354ef4-60ca-4bf6-89ac-ffaf94e3e294 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:14.704930 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9354ef4-60ca-4bf6-89ac-ffaf94e3e294 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:14.719605 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 227/459] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:14.720795 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9354ef4-60ca-4bf6-89ac-ffaf94e3e294 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:14.727245 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9062a6d5-a1eb-4622-a115-40375bfc920b None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:14.766042 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04ce810-dbc8-42ed-a2aa-9842899eda24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:14.766712 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ce810-dbc8-42ed-a2aa-9842899eda24 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:14.766991 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ce810-dbc8-42ed-a2aa-9842899eda24 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:14.767264 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04ce810-dbc8-42ed-a2aa-9842899eda24 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:14.775062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04ce810-dbc8-42ed-a2aa-9842899eda24 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:14.776561 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04ce810-dbc8-42ed-a2aa-9842899eda24 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:14.789003 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-c04ce810-dbc8-42ed-a2aa-9842899eda24 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 23 19:53:14.789003 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:14.789003 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:14.789003 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:14.789003 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:14.789003 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:14.789003 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:14.789003 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:14.789003 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:14.789003 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:14.789003 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:14.789003 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:14.789003 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 19:53:14.789003 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 19:53:14.789003 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 19:53:14.789003 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 19:53:14.789963 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:14.789963 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:14.789963 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 19:53:14.789963 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:14.789963 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 19:53:14.789963 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:14.789963 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:14.789963 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:14.789963 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:14.789963 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:14.789963 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:14.789963 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:14.789963 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:14.789963 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:14.789963 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:14.789963 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:14.789963 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:14.789963 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:14.790902 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:14.790902 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:14.790902 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 19:53:14.790902 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 19:53:14.790902 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 19:53:14.790902 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 19:53:14.790902 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 19:53:14.790902 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 19:53:14.790902 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 23 19:53:14.790902 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:14.790902 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 233/460] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:53:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:14.790902 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04ce810-dbc8-42ed-a2aa-9842899eda24 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:14.842709 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9354ef4-60ca-4bf6-89ac-ffaf94e3e294 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:14.878189 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63375c14-0ce6-471d-9815-3774a46d6125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:14.879015 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63375c14-0ce6-471d-9815-3774a46d6125 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:14.879400 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63375c14-0ce6-471d-9815-3774a46d6125 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:14.879751 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63375c14-0ce6-471d-9815-3774a46d6125 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:14.886554 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63375c14-0ce6-471d-9815-3774a46d6125 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:14.888116 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63375c14-0ce6-471d-9815-3774a46d6125 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:14.899942 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-63375c14-0ce6-471d-9815-3774a46d6125 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:14.899942 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:14.899942 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:14.899942 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:14.899942 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:14.899942 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:14.899942 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:14.899942 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:14.899942 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:14.899942 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:14.899942 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:14.899942 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:14.899942 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 19:53:14.899942 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 19:53:14.899942 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 19:53:14.900957 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 19:53:14.900957 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:14.900957 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:14.900957 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:14.900957 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:14.900957 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:14.900957 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:14.900957 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:14.900957 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:14.900957 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:14.900957 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:14.900957 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:14.900957 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:14.900957 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:14.900957 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:14.900957 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:14.900957 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:14.902028 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 19:53:14.902028 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 19:53:14.902028 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 19:53:14.902028 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 19:53:14.902028 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 19:53:14.902028 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 19:53:14.902028 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:14.902028 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:14.902565 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04ce810-dbc8-42ed-a2aa-9842899eda24 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:14.907961 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 228/461] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:53:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:14.907961 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63375c14-0ce6-471d-9815-3774a46d6125 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:14.927543 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:14.948354 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d47e14d-be34-4f06-9723-588da5ee93ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:14.949138 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47e14d-be34-4f06-9723-588da5ee93ec None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:14.949583 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47e14d-be34-4f06-9723-588da5ee93ec None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:14.949947 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d47e14d-be34-4f06-9723-588da5ee93ec None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:14.950686 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d47e14d-be34-4f06-9723-588da5ee93ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:14.951163 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d47e14d-be34-4f06-9723-588da5ee93ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:14.953142 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d47e14d-be34-4f06-9723-588da5ee93ec None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:14.963924 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:14.970406 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 234/462] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:14.971244 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d47e14d-be34-4f06-9723-588da5ee93ec None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:15.012492 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63375c14-0ce6-471d-9815-3774a46d6125 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:15.048014 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f76a24c-232c-4456-bcd0-da7ad8431bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:15.048568 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76a24c-232c-4456-bcd0-da7ad8431bc3 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:15.048857 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76a24c-232c-4456-bcd0-da7ad8431bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:15.049125 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f76a24c-232c-4456-bcd0-da7ad8431bc3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:15.057049 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f76a24c-232c-4456-bcd0-da7ad8431bc3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:15.058514 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f76a24c-232c-4456-bcd0-da7ad8431bc3 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:15.070981 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-3f76a24c-232c-4456-bcd0-da7ad8431bc3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 23 19:53:15.070981 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:15.070981 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:15.070981 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:15.070981 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:15.070981 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:15.070981 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:15.070981 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:15.070981 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:15.070981 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:15.070981 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:15.070981 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:15.070981 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 19:53:15.070981 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 19:53:15.070981 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 19:53:15.070981 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 19:53:15.072068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:15.072068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:15.072068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 19:53:15.072068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:15.072068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 19:53:15.072068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:15.072068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:15.072068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:15.072068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:15.072068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:15.072068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:15.072068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:15.072068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:15.072068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:15.072068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:15.072068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:15.072068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:15.072068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:15.073176 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:15.073176 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:15.073176 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 19:53:15.073176 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 19:53:15.073176 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 19:53:15.073176 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 19:53:15.073176 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 19:53:15.073176 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 19:53:15.073176 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 23 19:53:15.073176 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:15.073176 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 229/463] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:53:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:15.073176 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f76a24c-232c-4456-bcd0-da7ad8431bc3 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:15.090898 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d47e14d-be34-4f06-9723-588da5ee93ec None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:15.131370 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c125f-2c1b-45f0-a32c-73214f55d9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:15.131950 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c125f-2c1b-45f0-a32c-73214f55d9eb None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:15.132223 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c125f-2c1b-45f0-a32c-73214f55d9eb None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:15.132520 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c125f-2c1b-45f0-a32c-73214f55d9eb None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:15.133219 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c125f-2c1b-45f0-a32c-73214f55d9eb None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:15.133522 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c125f-2c1b-45f0-a32c-73214f55d9eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:15.134972 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c125f-2c1b-45f0-a32c-73214f55d9eb None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:15.157388 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 235/464] 10.222.0.13 () {56 vars in 1137 bytes} [Sun Oct 23 19:53:14 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:15.157388 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c125f-2c1b-45f0-a32c-73214f55d9eb None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:15.161663 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:15.186579 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f76a24c-232c-4456-bcd0-da7ad8431bc3 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:15.220345 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa76e55-17fe-40f2-b669-83f8f7601e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:15.220530 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:15.220950 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa76e55-17fe-40f2-b669-83f8f7601e8d None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:15.221239 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa76e55-17fe-40f2-b669-83f8f7601e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:15.221514 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa76e55-17fe-40f2-b669-83f8f7601e8d None admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:15.222343 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa76e55-17fe-40f2-b669-83f8f7601e8d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:15.222666 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa76e55-17fe-40f2-b669-83f8f7601e8d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:15.224167 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa76e55-17fe-40f2-b669-83f8f7601e8d None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:15.236565 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 230/465] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Oct 23 19:53:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:15.237358 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa76e55-17fe-40f2-b669-83f8f7601e8d None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:15.268429 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c125f-2c1b-45f0-a32c-73214f55d9eb None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:15.305435 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afecc6b-f775-411c-a4d6-c51396d9434a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:15.306407 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afecc6b-f775-411c-a4d6-c51396d9434a None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:15.306827 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afecc6b-f775-411c-a4d6-c51396d9434a None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:15.307241 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afecc6b-f775-411c-a4d6-c51396d9434a None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:15.315719 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afecc6b-f775-411c-a4d6-c51396d9434a None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:15.317552 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afecc6b-f775-411c-a4d6-c51396d9434a None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:15.331486 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-2afecc6b-f775-411c-a4d6-c51396d9434a None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 23 19:53:15.331486 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:15.331486 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:15.331486 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:15.331486 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:15.331486 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:15.331486 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:15.331486 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:15.331486 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:15.331486 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:15.331486 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:15.331486 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:15.331486 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 19:53:15.331486 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 19:53:15.331486 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 19:53:15.331486 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 19:53:15.332604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:15.332604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:15.332604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 19:53:15.332604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:15.332604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 19:53:15.332604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:15.332604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:15.332604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:15.332604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:15.332604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:15.332604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:15.332604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:15.332604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:15.332604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:15.332604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:15.332604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:15.332604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:15.332604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:15.333724 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:15.333724 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:15.333724 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 19:53:15.333724 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 19:53:15.333724 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 19:53:15.333724 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 19:53:15.333724 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 19:53:15.333724 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 19:53:15.333724 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 23 19:53:15.333724 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:15.336061 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 236/466] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Oct 23 19:53:15 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:15.337008 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afecc6b-f775-411c-a4d6-c51396d9434a None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:15.343953 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa76e55-17fe-40f2-b669-83f8f7601e8d None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:15.379395 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6810a367-fab9-4bf0-bed5-7be9197b0cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:15.380095 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6810a367-fab9-4bf0-bed5-7be9197b0cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:15.380852 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6810a367-fab9-4bf0-bed5-7be9197b0cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:15.381199 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6810a367-fab9-4bf0-bed5-7be9197b0cd2 None admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:15.382022 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6810a367-fab9-4bf0-bed5-7be9197b0cd2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:15.382771 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6810a367-fab9-4bf0-bed5-7be9197b0cd2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:15.384591 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6810a367-fab9-4bf0-bed5-7be9197b0cd2 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:15.393615 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:15.407572 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 231/467] 10.222.0.13 () {56 vars in 1137 bytes} [Sun Oct 23 19:53:15 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:15.407572 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6810a367-fab9-4bf0-bed5-7be9197b0cd2 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:15.450454 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afecc6b-f775-411c-a4d6-c51396d9434a None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:15.467836 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:15.486744 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a8dd77-3b7d-41f9-b973-7169940c3ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:15.487353 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8dd77-3b7d-41f9-b973-7169940c3ea2 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:15.487631 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8dd77-3b7d-41f9-b973-7169940c3ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:15.487899 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8dd77-3b7d-41f9-b973-7169940c3ea2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:15.510008 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6810a367-fab9-4bf0-bed5-7be9197b0cd2 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:15.549468 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45034d7-ac41-4661-824c-34042195c550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:15.550124 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45034d7-ac41-4661-824c-34042195c550 None admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:15.550518 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45034d7-ac41-4661-824c-34042195c550 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:15.550718 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45034d7-ac41-4661-824c-34042195c550 None admin] PATH_INFO: `/v3/projects/d40947427f5f43e7b6c82e3605737767/users/15f603d1e7f54751aa79317e2ea00a31/roles/5e0524ede3ee479c923c7b609a0ca402` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:15.561247 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a8dd77-3b7d-41f9-b973-7169940c3ea2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:15.563181 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a8dd77-3b7d-41f9-b973-7169940c3ea2 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:15.566505 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45034d7-ac41-4661-824c-34042195c550 None admin] RBAC: Authorizing `identity:create_grant(project_id=d40947427f5f43e7b6c82e3605737767, user_id=15f603d1e7f54751aa79317e2ea00a31, role_id=5e0524ede3ee479c923c7b609a0ca402)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:15.567924 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45034d7-ac41-4661-824c-34042195c550 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:15.581887 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 232/468] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Oct 23 19:53:15 2022] PUT /identity/v3/projects/d40947427f5f43e7b6c82e3605737767/users/15f603d1e7f54751aa79317e2ea00a31/roles/5e0524ede3ee479c923c7b609a0ca402 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 19:53:15.582793 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45034d7-ac41-4661-824c-34042195c550 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:15.658611 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:15.709754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 237/469] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Oct 23 19:53:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 19:53:15.709754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a8dd77-3b7d-41f9-b973-7169940c3ea2 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:15.730466 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:15.731109 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45034d7-ac41-4661-824c-34042195c550 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:15.774155 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb20397-e943-4a04-85a7-81f0fede5013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:15.774815 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb20397-e943-4a04-85a7-81f0fede5013 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:15.775543 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb20397-e943-4a04-85a7-81f0fede5013 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:15.775830 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb20397-e943-4a04-85a7-81f0fede5013 None admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:15.776574 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb20397-e943-4a04-85a7-81f0fede5013 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:15.776884 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb20397-e943-4a04-85a7-81f0fede5013 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:15.778372 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb20397-e943-4a04-85a7-81f0fede5013 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:15.798196 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 233/470] 10.222.0.13 () {56 vars in 1139 bytes} [Sun Oct 23 19:53:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:15.799069 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb20397-e943-4a04-85a7-81f0fede5013 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:15.836599 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a8dd77-3b7d-41f9-b973-7169940c3ea2 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:15.875342 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab29cc-d307-4b08-9df5-8a1c8c34da0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:15.875955 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab29cc-d307-4b08-9df5-8a1c8c34da0d None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:15.876230 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab29cc-d307-4b08-9df5-8a1c8c34da0d None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:15.876773 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edab29cc-d307-4b08-9df5-8a1c8c34da0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:15.930584 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb20397-e943-4a04-85a7-81f0fede5013 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:15.941630 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:15.956981 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab29cc-d307-4b08-9df5-8a1c8c34da0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:15.958576 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edab29cc-d307-4b08-9df5-8a1c8c34da0d None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:15.977655 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:15.982116 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09e908a-891d-444a-b8ae-8bea35e0fd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:15.983441 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e908a-891d-444a-b8ae-8bea35e0fd35 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:15.984659 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e908a-891d-444a-b8ae-8bea35e0fd35 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:15.985089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e908a-891d-444a-b8ae-8bea35e0fd35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:15.996846 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09e908a-891d-444a-b8ae-8bea35e0fd35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:15.998512 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09e908a-891d-444a-b8ae-8bea35e0fd35 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:16.014873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-b09e908a-891d-444a-b8ae-8bea35e0fd35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:16.014873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:16.014873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:16.014873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:16.014873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:16.014873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:16.014873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:16.014873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:16.014873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:16.014873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:16.014873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:16.014873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:16.014873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:16.014873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:16.014873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:16.014873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:16.016344 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:16.016344 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:16.016344 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:16.016344 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:16.016344 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:16.016344 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:16.016344 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:16.016344 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:16.016344 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:16.016344 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:16.016344 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:16.016344 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:16.016344 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:16.016344 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:16.016344 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:16.016344 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:16.016344 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:16.017191 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:16.017191 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:16.017191 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:16.019184 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 234/471] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:16.020149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09e908a-891d-444a-b8ae-8bea35e0fd35 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:16.075131 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 238/472] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Oct 23 19:53:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 19:53:16.076370 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab29cc-d307-4b08-9df5-8a1c8c34da0d None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:16.163853 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09e908a-891d-444a-b8ae-8bea35e0fd35 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:16.181556 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edab29cc-d307-4b08-9df5-8a1c8c34da0d None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:16.203351 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:16.218531 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f7054-20ca-4193-bcb0-839cf6c82c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:16.219249 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f7054-20ca-4193-bcb0-839cf6c82c8b None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:16.219637 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f7054-20ca-4193-bcb0-839cf6c82c8b None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:16.220008 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f7054-20ca-4193-bcb0-839cf6c82c8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:16.221859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f40b2-7e02-4498-8fd9-a9131782b88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:16.222616 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f40b2-7e02-4498-8fd9-a9131782b88c None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:16.222959 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f40b2-7e02-4498-8fd9-a9131782b88c None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:16.223338 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69f40b2-7e02-4498-8fd9-a9131782b88c None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:16.224044 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f40b2-7e02-4498-8fd9-a9131782b88c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:16.224412 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f40b2-7e02-4498-8fd9-a9131782b88c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:16.225853 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69f40b2-7e02-4498-8fd9-a9131782b88c None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:16.237940 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 239/473] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:16.239017 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f40b2-7e02-4498-8fd9-a9131782b88c None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:16.259761 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:16.316988 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f7054-20ca-4193-bcb0-839cf6c82c8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:16.319014 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4f7054-20ca-4193-bcb0-839cf6c82c8b None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:16.353771 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69f40b2-7e02-4498-8fd9-a9131782b88c None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:16.393769 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380b4a9-8b98-4d26-abe0-6ede31ee5665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:16.394511 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380b4a9-8b98-4d26-abe0-6ede31ee5665 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:16.394791 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380b4a9-8b98-4d26-abe0-6ede31ee5665 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:16.395060 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380b4a9-8b98-4d26-abe0-6ede31ee5665 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:16.407345 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380b4a9-8b98-4d26-abe0-6ede31ee5665 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:16.408886 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380b4a9-8b98-4d26-abe0-6ede31ee5665 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:16.424644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-9380b4a9-8b98-4d26-abe0-6ede31ee5665 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:16.424644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:16.424644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:16.424644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:16.424644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:16.424644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:16.424644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:16.424644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:16.424644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:16.424644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:16.424644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:16.424644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:16.424644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:16.424644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:16.424644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:16.424644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:16.425558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:16.425558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:16.425558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:16.425558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:16.425558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:16.425558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:16.425558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:16.425558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:16.425558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:16.425558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:16.425558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:16.425558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:16.425558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:16.425558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:16.425558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:16.425558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:16.425558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:16.426432 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:16.426432 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:16.426432 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:16.426432 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 240/474] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:16.427082 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380b4a9-8b98-4d26-abe0-6ede31ee5665 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:16.457243 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 235/475] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Oct 23 19:53:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 19:53:16.459444 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f7054-20ca-4193-bcb0-839cf6c82c8b None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:16.461432 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:16.531834 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:16.544949 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380b4a9-8b98-4d26-abe0-6ede31ee5665 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:16.579405 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f7054-20ca-4193-bcb0-839cf6c82c8b None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:16.582179 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a301d0-0a11-4b22-863f-a4b163934ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:16.583318 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a301d0-0a11-4b22-863f-a4b163934ade None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:16.583318 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a301d0-0a11-4b22-863f-a4b163934ade None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:16.583886 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a301d0-0a11-4b22-863f-a4b163934ade None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:16.591239 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a301d0-0a11-4b22-863f-a4b163934ade None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:16.592884 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a301d0-0a11-4b22-863f-a4b163934ade None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:16.605146 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-65a301d0-0a11-4b22-863f-a4b163934ade None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:16.605146 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:16.605146 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:16.605146 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:16.605146 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:16.605146 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:16.605146 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:16.605146 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:16.605146 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:16.605146 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:16.605146 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:16.605146 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:16.605146 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 19:53:16.605146 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 19:53:16.605146 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 19:53:16.605997 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 19:53:16.605997 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:16.605997 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:16.605997 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:16.605997 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:16.605997 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:16.605997 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:16.605997 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:16.605997 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:16.605997 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:16.605997 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:16.605997 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:16.605997 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:16.605997 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:16.605997 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:16.605997 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:16.605997 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:16.606874 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 19:53:16.606874 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 19:53:16.606874 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 19:53:16.606874 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 19:53:16.606874 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 19:53:16.606874 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 19:53:16.606874 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:16.606874 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:16.611953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 241/476] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:53:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:16.613100 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a301d0-0a11-4b22-863f-a4b163934ade None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:16.621126 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f8635-dc15-4bb0-96cd-52544c9c4e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:16.622058 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f8635-dc15-4bb0-96cd-52544c9c4e0e None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:16.622622 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f8635-dc15-4bb0-96cd-52544c9c4e0e None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:16.623099 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f8635-dc15-4bb0-96cd-52544c9c4e0e None admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:16.623969 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507f8635-dc15-4bb0-96cd-52544c9c4e0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:16.624588 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507f8635-dc15-4bb0-96cd-52544c9c4e0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:16.626292 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507f8635-dc15-4bb0-96cd-52544c9c4e0e None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:16.639086 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 236/477] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:16.640763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f8635-dc15-4bb0-96cd-52544c9c4e0e None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:16.721448 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:16.740000 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a301d0-0a11-4b22-863f-a4b163934ade None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:16.779406 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f8635-dc15-4bb0-96cd-52544c9c4e0e None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:16.788988 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb81a19-961c-489e-8f49-60e86d2cd2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:16.789925 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb81a19-961c-489e-8f49-60e86d2cd2bb None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:16.790789 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb81a19-961c-489e-8f49-60e86d2cd2bb None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:16.791310 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb81a19-961c-489e-8f49-60e86d2cd2bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:16.799780 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:16.807207 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb81a19-961c-489e-8f49-60e86d2cd2bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:16.809423 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb81a19-961c-489e-8f49-60e86d2cd2bb None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:16.822188 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affeb2c6-38ed-40f0-a78a-046c63a593ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:16.822903 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affeb2c6-38ed-40f0-a78a-046c63a593ae None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:16.823357 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affeb2c6-38ed-40f0-a78a-046c63a593ae None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:16.823787 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affeb2c6-38ed-40f0-a78a-046c63a593ae None admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:16.824532 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affeb2c6-38ed-40f0-a78a-046c63a593ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:16.824873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affeb2c6-38ed-40f0-a78a-046c63a593ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:16.826540 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affeb2c6-38ed-40f0-a78a-046c63a593ae None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:16.830701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-cfb81a19-961c-489e-8f49-60e86d2cd2bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:16.830701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:16.830701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:16.830701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:16.830701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:16.830701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:16.830701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:16.830701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:16.830701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:16.830701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:16.830701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:16.830701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:16.830701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:16.830701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:16.830701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:16.830701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:16.831563 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:16.831563 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:16.831563 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:16.831563 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:16.831563 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:16.831563 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:16.831563 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:16.831563 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:16.831563 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:16.831563 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:16.831563 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:16.831563 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:16.831563 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:16.831563 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:16.831563 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:16.831563 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:16.831563 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:16.832407 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:16.832407 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:16.832407 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:16.834995 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 242/478] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:16.836450 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb81a19-961c-489e-8f49-60e86d2cd2bb None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:16.840342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 237/479] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Oct 23 19:53:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:16.842619 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a41d1-8787-4063-bcf8-c68e96686ba2 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:16.842950 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a41d1-8787-4063-bcf8-c68e96686ba2 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:16.843292 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a41d1-8787-4063-bcf8-c68e96686ba2 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:16.844753 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 238/480] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:16.845529 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a41d1-8787-4063-bcf8-c68e96686ba2 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:16.953398 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb81a19-961c-489e-8f49-60e86d2cd2bb None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:16.955346 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a41d1-8787-4063-bcf8-c68e96686ba2 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:16.990034 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a05df-4b71-4298-9277-9081c5414a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:16.990467 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e29bb7a-327d-4afb-890e-6c2325155d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:16.990596 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a05df-4b71-4298-9277-9081c5414a2e None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:16.990864 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a05df-4b71-4298-9277-9081c5414a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:16.991087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29bb7a-327d-4afb-890e-6c2325155d0b None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:16.991195 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a05df-4b71-4298-9277-9081c5414a2e None admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:16.991359 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29bb7a-327d-4afb-890e-6c2325155d0b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:16.991625 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e29bb7a-327d-4afb-890e-6c2325155d0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:16.991781 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268a05df-4b71-4298-9277-9081c5414a2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:16.992056 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268a05df-4b71-4298-9277-9081c5414a2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:16.993542 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268a05df-4b71-4298-9277-9081c5414a2e None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:16.999184 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29bb7a-327d-4afb-890e-6c2325155d0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:17.001336 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e29bb7a-327d-4afb-890e-6c2325155d0b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:17.005435 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 239/481] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:17.010425 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a05df-4b71-4298-9277-9081c5414a2e None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:17.017272 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-5e29bb7a-327d-4afb-890e-6c2325155d0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:17.017272 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:17.017272 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:17.017272 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:17.017272 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:17.017272 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:17.017272 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:17.017272 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:17.017272 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:17.017272 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:17.017272 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:17.017272 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:17.017272 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 19:53:17.017272 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 19:53:17.017272 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 19:53:17.018144 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 19:53:17.018144 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:17.018144 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:17.018144 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:17.018144 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:17.018144 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:17.018144 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:17.018144 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:17.018144 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:17.018144 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:17.018144 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:17.018144 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:17.018144 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:17.018144 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:17.018144 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:17.018144 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:17.018144 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:17.019013 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 19:53:17.019013 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 19:53:17.019013 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 19:53:17.019013 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 19:53:17.019013 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 19:53:17.019013 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 19:53:17.019013 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:17.019013 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:17.025951 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 243/482] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:53:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:17.025951 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24cb306-ae5d-4312-96f4-d5a91187f33e None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:17.025951 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24cb306-ae5d-4312-96f4-d5a91187f33e None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:17.025951 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24cb306-ae5d-4312-96f4-d5a91187f33e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:17.032805 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:17.045877 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-d24cb306-ae5d-4312-96f4-d5a91187f33e None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:17.047805 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d24cb306-ae5d-4312-96f4-d5a91187f33e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:17.052714 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:17.117582 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a05df-4b71-4298-9277-9081c5414a2e None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:17.120617 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 244/483] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:53:17.121358 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24cb306-ae5d-4312-96f4-d5a91187f33e None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:17.160247 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8621e-f9ec-4e9e-a130-813cbfd5c346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:17.161220 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8621e-f9ec-4e9e-a130-813cbfd5c346 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:17.161707 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8621e-f9ec-4e9e-a130-813cbfd5c346 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:17.162180 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8621e-f9ec-4e9e-a130-813cbfd5c346 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:17.163153 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8621e-f9ec-4e9e-a130-813cbfd5c346 None admin] RBAC: Adding query filter params (role.id=5e0524ede3ee479c923c7b609a0ca402, scope.project.id=d40947427f5f43e7b6c82e3605737767, user.id=021216f90d7f4ff6b6eb7a1d86399d99) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:17.163663 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8621e-f9ec-4e9e-a130-813cbfd5c346 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:17.165811 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8621e-f9ec-4e9e-a130-813cbfd5c346 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:17.174862 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 240/484] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Oct 23 19:53:17 2022] GET /identity/v3/role_assignments?user.id=021216f90d7f4ff6b6eb7a1d86399d99&scope.project.id=d40947427f5f43e7b6c82e3605737767&role.id=5e0524ede3ee479c923c7b609a0ca402 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:17.176174 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8621e-f9ec-4e9e-a130-813cbfd5c346 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:17.236246 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24cb306-ae5d-4312-96f4-d5a91187f33e None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:17.278661 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:17.285656 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6cabe-e3b0-4010-b929-f7631ae8d4fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:17.287159 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6cabe-e3b0-4010-b929-f7631ae8d4fc None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:17.287925 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6cabe-e3b0-4010-b929-f7631ae8d4fc None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:17.288395 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b6cabe-e3b0-4010-b929-f7631ae8d4fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:17.299101 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b6cabe-e3b0-4010-b929-f7631ae8d4fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:17.301552 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b6cabe-e3b0-4010-b929-f7631ae8d4fc None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:17.309325 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8621e-f9ec-4e9e-a130-813cbfd5c346 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:17.311397 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:17.318296 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-10b6cabe-e3b0-4010-b929-f7631ae8d4fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:17.318296 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:17.318296 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:17.318296 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:17.318296 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:17.318296 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:17.318296 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:17.318296 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:17.318296 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:17.318296 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:17.318296 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:17.318296 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:17.318296 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 19:53:17.318296 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 19:53:17.318296 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 19:53:17.319256 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 19:53:17.319256 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:17.319256 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:17.319256 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:17.319256 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:17.319256 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:17.319256 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:17.319256 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:17.319256 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:17.319256 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:17.319256 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:17.319256 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:17.319256 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:17.319256 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:17.319256 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:17.319256 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:17.319256 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:17.320234 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 19:53:17.320234 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 19:53:17.320234 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 19:53:17.320234 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 19:53:17.320234 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 19:53:17.320234 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 19:53:17.320234 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:17.320234 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:17.323036 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 245/485] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:53:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:17.328904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6cabe-e3b0-4010-b929-f7631ae8d4fc None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:17.356583 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c971f6-35e5-4b40-aa8b-ab0603455429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:17.357390 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c971f6-35e5-4b40-aa8b-ab0603455429 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:17.357764 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c971f6-35e5-4b40-aa8b-ab0603455429 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:17.358134 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c971f6-35e5-4b40-aa8b-ab0603455429 None admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:17.359526 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c971f6-35e5-4b40-aa8b-ab0603455429 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:17.359931 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c971f6-35e5-4b40-aa8b-ab0603455429 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:17.362151 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c971f6-35e5-4b40-aa8b-ab0603455429 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:17.381562 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 241/486] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Oct 23 19:53:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:17.389754 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c971f6-35e5-4b40-aa8b-ab0603455429 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:17.510906 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c971f6-35e5-4b40-aa8b-ab0603455429 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:17.516815 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b6cabe-e3b0-4010-b929-f7631ae8d4fc None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:17.562337 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebffdc0-d827-44b6-bafa-745c0d454472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:17.563379 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebffdc0-d827-44b6-bafa-745c0d454472 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:17.563913 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebffdc0-d827-44b6-bafa-745c0d454472 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:17.564187 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebffdc0-d827-44b6-bafa-745c0d454472 None admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:17.564892 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebffdc0-d827-44b6-bafa-745c0d454472 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:17.565194 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebffdc0-d827-44b6-bafa-745c0d454472 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:17.566672 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebffdc0-d827-44b6-bafa-745c0d454472 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:17.567964 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd549f0-ec7e-4193-8008-30ef3c12a0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:17.568584 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd549f0-ec7e-4193-8008-30ef3c12a0f5 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:17.568862 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd549f0-ec7e-4193-8008-30ef3c12a0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:17.569132 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd549f0-ec7e-4193-8008-30ef3c12a0f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:17.569860 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd549f0-ec7e-4193-8008-30ef3c12a0f5 None admin] RBAC: Adding query filter params (role.id=5e0524ede3ee479c923c7b609a0ca402, scope.project.id=d40947427f5f43e7b6c82e3605737767, user.id=7bcc5f5101fb4913aaf5caf564ca6b60) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:17.570174 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd549f0-ec7e-4193-8008-30ef3c12a0f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:17.571628 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd549f0-ec7e-4193-8008-30ef3c12a0f5 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:17.579709 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 242/487] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Oct 23 19:53:17 2022] GET /identity/v3/role_assignments?user.id=7bcc5f5101fb4913aaf5caf564ca6b60&scope.project.id=d40947427f5f43e7b6c82e3605737767&role.id=5e0524ede3ee479c923c7b609a0ca402 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:17.580354 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd549f0-ec7e-4193-8008-30ef3c12a0f5 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:17.582845 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 246/488] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Oct 23 19:53:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:53:17.589865 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebffdc0-d827-44b6-bafa-745c0d454472 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:17.618284 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:17.660038 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:17.714782 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd549f0-ec7e-4193-8008-30ef3c12a0f5 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:17.737782 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebffdc0-d827-44b6-bafa-745c0d454472 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:17.782093 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c899e-0492-4620-84b3-a942d2fb0be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:17.782776 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c899e-0492-4620-84b3-a942d2fb0be2 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:17.783055 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c899e-0492-4620-84b3-a942d2fb0be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:17.783323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9c899e-0492-4620-84b3-a942d2fb0be2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:17.803485 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9c899e-0492-4620-84b3-a942d2fb0be2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:53:17.809542 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-7d9c899e-0492-4620-84b3-a942d2fb0be2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:53:17.809542 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:17.809542 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:17.809542 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:17.809542 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:17.809542 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:17.809542 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:17.809542 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:17.809542 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:17.809542 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:17.809542 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:17.809542 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:17.809542 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:53:17.809542 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:53:17.809542 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:53:17.809542 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:53:17.810559 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:53:17.810559 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:53:17.810559 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:53:17.810559 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:53:17.810559 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:17.810559 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:17.810559 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:17.810559 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:17.810559 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:17.810559 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:17.810559 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:17.810559 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:17.810559 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:17.810559 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:17.810559 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:17.810559 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:17.811347 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:17.811347 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:17.811347 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:17.811347 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:17.811347 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:53:17.811347 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:53:17.811347 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:53:17.811347 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:53:17.811347 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:53:17.811347 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:53:17.811347 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:53:17.811347 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:17.815210 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e673d993-5a60-4828-8b61-acad9963ff73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:17.815773 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673d993-5a60-4828-8b61-acad9963ff73 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:17.816124 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673d993-5a60-4828-8b61-acad9963ff73 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:17.816398 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e673d993-5a60-4828-8b61-acad9963ff73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:17.817206 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e673d993-5a60-4828-8b61-acad9963ff73 None admin] RBAC: Adding query filter params (role.id=5e0524ede3ee479c923c7b609a0ca402, scope.project.id=d40947427f5f43e7b6c82e3605737767, user.id=ba8ff3413bc549d29486d1c724a1f383) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:17.817511 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e673d993-5a60-4828-8b61-acad9963ff73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:17.819225 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e673d993-5a60-4828-8b61-acad9963ff73 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:17.821241 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 243/489] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Oct 23 19:53:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:17.822179 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c899e-0492-4620-84b3-a942d2fb0be2 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:17.829537 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 247/490] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Oct 23 19:53:17 2022] GET /identity/v3/role_assignments?user.id=ba8ff3413bc549d29486d1c724a1f383&scope.project.id=d40947427f5f43e7b6c82e3605737767&role.id=5e0524ede3ee479c923c7b609a0ca402 => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:17.831153 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b712c2-04c0-452d-b7fa-9f222d62edfb None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:17.831645 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b712c2-04c0-452d-b7fa-9f222d62edfb None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:17.832124 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b712c2-04c0-452d-b7fa-9f222d62edfb None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:17.833695 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 248/491] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:17.840146 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc693d-058c-4539-ba22-8b442e6332bb None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:17.840588 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc693d-058c-4539-ba22-8b442e6332bb None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:17.840976 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc693d-058c-4539-ba22-8b442e6332bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:17.864851 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-37dc693d-058c-4539-ba22-8b442e6332bb None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:17.866351 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-37dc693d-058c-4539-ba22-8b442e6332bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:17.943481 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:17.960655 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 249/492] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 19:53:17.969458 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9c899e-0492-4620-84b3-a942d2fb0be2 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:18.009844 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e420f1-90f8-443f-a362-69b4f2623f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:18.010839 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e420f1-90f8-443f-a362-69b4f2623f4f None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:18.011374 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e420f1-90f8-443f-a362-69b4f2623f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:18.011845 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e420f1-90f8-443f-a362-69b4f2623f4f None admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:18.012808 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e420f1-90f8-443f-a362-69b4f2623f4f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:18.013308 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e420f1-90f8-443f-a362-69b4f2623f4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:18.014916 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e420f1-90f8-443f-a362-69b4f2623f4f None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:18.025721 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 244/493] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:53:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:18.030992 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc693d-058c-4539-ba22-8b442e6332bb None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:18.083878 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:18.144002 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc693d-058c-4539-ba22-8b442e6332bb None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:18.184340 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e281c-8032-43d8-bf37-734371af47ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:18.185357 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e281c-8032-43d8-bf37-734371af47ae None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:18.185836 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e281c-8032-43d8-bf37-734371af47ae None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:18.186363 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130e281c-8032-43d8-bf37-734371af47ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:18.199363 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e281c-8032-43d8-bf37-734371af47ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:18.201112 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130e281c-8032-43d8-bf37-734371af47ae None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:18.222917 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-130e281c-8032-43d8-bf37-734371af47ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:18.222917 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:18.222917 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:18.222917 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:18.222917 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:18.222917 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:18.222917 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:18.222917 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:18.222917 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:18.222917 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:18.222917 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:18.222917 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:18.222917 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:18.222917 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:18.222917 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:18.222917 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:18.223857 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:18.223857 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:18.223857 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:18.223857 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:18.223857 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:18.223857 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:18.223857 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:18.223857 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:18.223857 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:18.223857 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:18.223857 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:18.223857 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:18.223857 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:18.223857 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:18.223857 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:18.223857 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:18.223857 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:18.225365 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:18.225365 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:18.225365 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:18.227545 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 250/494] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:53:18.233686 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e420f1-90f8-443f-a362-69b4f2623f4f None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:18.259506 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e281c-8032-43d8-bf37-734371af47ae None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:18.387577 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e420f1-90f8-443f-a362-69b4f2623f4f None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:18.415649 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:18.425442 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359588ee-d15b-4fb6-b0dc-6c2c3637cc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:18.426382 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359588ee-d15b-4fb6-b0dc-6c2c3637cc21 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:18.426692 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130e281c-8032-43d8-bf37-734371af47ae None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:18.427243 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359588ee-d15b-4fb6-b0dc-6c2c3637cc21 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:18.427724 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359588ee-d15b-4fb6-b0dc-6c2c3637cc21 None admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:18.428571 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359588ee-d15b-4fb6-b0dc-6c2c3637cc21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:18.429234 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359588ee-d15b-4fb6-b0dc-6c2c3637cc21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:18.430910 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359588ee-d15b-4fb6-b0dc-6c2c3637cc21 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:18.446088 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 245/495] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:18.452789 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359588ee-d15b-4fb6-b0dc-6c2c3637cc21 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:18.464695 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:18.551599 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f92dfbf-98ef-4041-ab3b-53ebfebb2288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:18.552325 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92dfbf-98ef-4041-ab3b-53ebfebb2288 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:18.552520 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92dfbf-98ef-4041-ab3b-53ebfebb2288 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:18.552794 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f92dfbf-98ef-4041-ab3b-53ebfebb2288 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:18.570838 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359588ee-d15b-4fb6-b0dc-6c2c3637cc21 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:18.598856 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f92dfbf-98ef-4041-ab3b-53ebfebb2288 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:53:18.604149 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-7f92dfbf-98ef-4041-ab3b-53ebfebb2288 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 19:53:18.604149 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:18.604149 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:18.604149 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:18.604149 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:18.604149 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:18.604149 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:18.604149 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:18.604149 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:18.604149 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:18.604149 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:18.604149 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:18.604149 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:53:18.604149 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:53:18.604149 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:53:18.604149 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:53:18.605033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:53:18.605033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:53:18.605033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:53:18.605033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:53:18.605033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:18.605033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:18.605033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:18.605033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:18.605033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:18.605033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:18.605033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:18.605033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:18.605033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:18.605033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:18.605033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:18.605033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:18.605916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:18.605916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:18.605916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:18.605916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:18.605916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:53:18.605916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:53:18.605916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:53:18.605916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:53:18.605916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:53:18.605916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:53:18.605916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 19:53:18.605916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:18.607110 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 251/496] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Oct 23 19:53:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 347 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:53:18.610026 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a516dbf-16ad-4c7d-80c1-fc81ef47d4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:18.610630 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a516dbf-16ad-4c7d-80c1-fc81ef47d4c8 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:18.610958 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a516dbf-16ad-4c7d-80c1-fc81ef47d4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:18.611234 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a516dbf-16ad-4c7d-80c1-fc81ef47d4c8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:18.613971 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f92dfbf-98ef-4041-ab3b-53ebfebb2288 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:18.632388 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a516dbf-16ad-4c7d-80c1-fc81ef47d4c8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:18.636751 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a516dbf-16ad-4c7d-80c1-fc81ef47d4c8 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:18.659010 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-3a516dbf-16ad-4c7d-80c1-fc81ef47d4c8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 23 19:53:18.659010 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:18.659010 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:18.659010 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:18.659010 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:18.659010 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:18.659010 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:18.659010 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:18.659010 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:18.659010 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:18.659010 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:18.659010 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:18.659010 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 19:53:18.659010 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 19:53:18.659010 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 19:53:18.659010 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 19:53:18.661140 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:18.661140 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:18.661140 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 19:53:18.661140 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:18.661140 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 19:53:18.661140 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:18.661140 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:18.661140 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:18.661140 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:18.661140 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:18.661140 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:18.661140 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:18.661140 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:18.661140 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:18.661140 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:18.661140 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:18.661140 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:18.661140 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:18.663354 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:18.663354 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:18.663354 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 19:53:18.663354 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 19:53:18.663354 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 19:53:18.663354 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 19:53:18.663354 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 19:53:18.663354 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 19:53:18.663354 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 23 19:53:18.663354 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:18.668458 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 246/497] 10.222.0.13 () {56 vars in 1093 bytes} [Sun Oct 23 19:53:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:18.672647 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a516dbf-16ad-4c7d-80c1-fc81ef47d4c8 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:18.780165 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:18.819066 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a516dbf-16ad-4c7d-80c1-fc81ef47d4c8 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:18.856454 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1383ba-68ec-4678-b243-fc3aeeb446e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:18.857072 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1383ba-68ec-4678-b243-fc3aeeb446e8 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:18.857354 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1383ba-68ec-4678-b243-fc3aeeb446e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:18.857629 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1383ba-68ec-4678-b243-fc3aeeb446e8 None admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:18.858399 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1383ba-68ec-4678-b243-fc3aeeb446e8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:18.859246 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1383ba-68ec-4678-b243-fc3aeeb446e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:18.860944 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1383ba-68ec-4678-b243-fc3aeeb446e8 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:18.868207 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f92dfbf-98ef-4041-ab3b-53ebfebb2288 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:18.881784 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 247/498] 10.222.0.13 () {56 vars in 1133 bytes} [Sun Oct 23 19:53:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:18.888534 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1383ba-68ec-4678-b243-fc3aeeb446e8 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:18.919126 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e46d3-3e05-43e4-b18a-61c58eec24e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:18.919661 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e46d3-3e05-43e4-b18a-61c58eec24e4 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:18.919935 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e46d3-3e05-43e4-b18a-61c58eec24e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:18.920206 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e46d3-3e05-43e4-b18a-61c58eec24e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:18.920872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e46d3-3e05-43e4-b18a-61c58eec24e4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:18.921688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e46d3-3e05-43e4-b18a-61c58eec24e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:18.924628 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e46d3-3e05-43e4-b18a-61c58eec24e4 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:18.942759 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 252/499] 10.222.0.13 () {56 vars in 1103 bytes} [Sun Oct 23 19:53:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:18.949483 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e46d3-3e05-43e4-b18a-61c58eec24e4 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:18.993687 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:19.055131 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1383ba-68ec-4678-b243-fc3aeeb446e8 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:19.107447 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:19.122581 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9d760-573a-4e9c-a156-16fcb10f394a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:19.123220 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9d760-573a-4e9c-a156-16fcb10f394a None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:19.123503 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9d760-573a-4e9c-a156-16fcb10f394a None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:19.123782 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d9d760-573a-4e9c-a156-16fcb10f394a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:19.144186 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d9d760-573a-4e9c-a156-16fcb10f394a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:19.145643 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d9d760-573a-4e9c-a156-16fcb10f394a None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:19.150924 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e46d3-3e05-43e4-b18a-61c58eec24e4 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:19.173791 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-c5d9d760-573a-4e9c-a156-16fcb10f394a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:19.173791 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:19.173791 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:19.173791 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:19.173791 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:19.173791 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:19.173791 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:19.173791 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:19.173791 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:19.173791 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:19.173791 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:19.173791 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:19.173791 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:19.173791 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:19.173791 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:19.173791 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:19.174733 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:19.174733 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:19.174733 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:19.174733 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:19.174733 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:19.174733 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:19.174733 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:19.174733 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:19.174733 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:19.174733 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:19.174733 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:19.174733 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:19.174733 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:19.174733 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:19.174733 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:19.174733 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:19.174733 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:19.176630 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:19.176630 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:19.176630 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:19.176630 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 248/500] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:19.180559 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9d760-573a-4e9c-a156-16fcb10f394a None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:19.215034 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c5d57-5c71-4776-8a13-da5f3978de9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:19.215682 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c5d57-5c71-4776-8a13-da5f3978de9a None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:19.215963 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c5d57-5c71-4776-8a13-da5f3978de9a None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:19.216237 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c5d57-5c71-4776-8a13-da5f3978de9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:19.235708 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54c5d57-5c71-4776-8a13-da5f3978de9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:19.237176 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54c5d57-5c71-4776-8a13-da5f3978de9a None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:19.265668 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-e54c5d57-5c71-4776-8a13-da5f3978de9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:19.265668 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:19.265668 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:19.265668 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:19.265668 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:19.265668 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:19.265668 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:19.265668 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:19.265668 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:19.265668 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:19.265668 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:19.265668 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:19.265668 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:19.265668 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:19.265668 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:19.265668 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:19.266701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:19.266701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:19.266701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:19.266701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:19.266701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:19.266701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:19.266701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:19.266701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:19.266701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:19.266701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:19.266701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:19.266701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:19.266701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:19.266701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:19.266701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:19.266701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:19.266701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:19.268189 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:19.268189 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:19.268189 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:19.268189 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 253/501] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:53:19.272500 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c5d57-5c71-4776-8a13-da5f3978de9a None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:19.379937 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d9d760-573a-4e9c-a156-16fcb10f394a None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:19.394684 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:19.429995 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c5d57-5c71-4776-8a13-da5f3978de9a None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:19.454908 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcebe9bb-56f4-4209-bb9a-595b76c6da36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:19.455526 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcebe9bb-56f4-4209-bb9a-595b76c6da36 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:19.455807 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcebe9bb-56f4-4209-bb9a-595b76c6da36 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:19.456079 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcebe9bb-56f4-4209-bb9a-595b76c6da36 None admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:19.456725 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcebe9bb-56f4-4209-bb9a-595b76c6da36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:19.457027 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcebe9bb-56f4-4209-bb9a-595b76c6da36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:19.458420 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcebe9bb-56f4-4209-bb9a-595b76c6da36 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:19.485855 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 249/502] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:19.491872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf911aa-f530-487c-8050-98a63a873173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:19.492497 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf911aa-f530-487c-8050-98a63a873173 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:19.492778 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf911aa-f530-487c-8050-98a63a873173 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:19.493832 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcebe9bb-56f4-4209-bb9a-595b76c6da36 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:19.494385 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf911aa-f530-487c-8050-98a63a873173 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:19.495043 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf911aa-f530-487c-8050-98a63a873173 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:19.495410 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf911aa-f530-487c-8050-98a63a873173 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:19.496778 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf911aa-f530-487c-8050-98a63a873173 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:19.520340 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 254/503] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:53:19.526492 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf911aa-f530-487c-8050-98a63a873173 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:19.559429 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:19.659436 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcebe9bb-56f4-4209-bb9a-595b76c6da36 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:19.721419 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf911aa-f530-487c-8050-98a63a873173 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:19.732345 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0321a2d-aaac-4e59-883d-c2e7669ec7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:19.733002 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0321a2d-aaac-4e59-883d-c2e7669ec7d7 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:19.733279 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0321a2d-aaac-4e59-883d-c2e7669ec7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:19.733551 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0321a2d-aaac-4e59-883d-c2e7669ec7d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:19.751339 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0321a2d-aaac-4e59-883d-c2e7669ec7d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:19.752801 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0321a2d-aaac-4e59-883d-c2e7669ec7d7 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:19.754175 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:19.760141 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe3b5f-415f-4b62-aa92-41890e919a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:19.760823 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe3b5f-415f-4b62-aa92-41890e919a40 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:19.761107 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe3b5f-415f-4b62-aa92-41890e919a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:19.761380 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fe3b5f-415f-4b62-aa92-41890e919a40 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:19.769120 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe3b5f-415f-4b62-aa92-41890e919a40 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:19.770595 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fe3b5f-415f-4b62-aa92-41890e919a40 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:19.776216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-f0321a2d-aaac-4e59-883d-c2e7669ec7d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:19.776216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:19.776216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:19.776216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:19.776216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:19.776216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:19.776216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:19.776216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:19.776216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:19.776216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:19.776216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:19.776216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:19.776216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 19:53:19.776216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 19:53:19.776216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 19:53:19.777168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 19:53:19.777168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:19.777168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:19.777168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:19.777168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:19.777168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:19.777168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:19.777168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:19.777168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:19.777168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:19.777168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:19.777168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:19.777168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:19.777168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:19.777168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:19.777168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:19.777168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:19.778064 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 19:53:19.778064 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 19:53:19.778064 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 19:53:19.778064 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 19:53:19.778064 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 19:53:19.778064 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 19:53:19.778064 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:19.778064 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:19.782011 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 250/504] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:53:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:19.782902 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0321a2d-aaac-4e59-883d-c2e7669ec7d7 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:19.783115 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-08fe3b5f-415f-4b62-aa92-41890e919a40 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 23 19:53:19.783115 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:19.783115 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:19.783115 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:19.783115 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:19.783115 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:19.783115 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:19.783115 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:19.783115 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:19.783115 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:19.783115 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:19.783115 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:19.783115 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 19:53:19.783115 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 19:53:19.783115 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 19:53:19.783115 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 19:53:19.784372 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:19.784372 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:19.784372 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 19:53:19.784372 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:19.784372 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 19:53:19.784372 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:19.784372 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:19.784372 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:19.784372 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:19.784372 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:19.784372 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:19.784372 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:19.784372 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:19.784372 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:19.784372 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:19.784372 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:19.784372 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:19.784372 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:19.785766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:19.785766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:19.785766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 19:53:19.785766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 19:53:19.785766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 19:53:19.785766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 19:53:19.785766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 19:53:19.785766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 19:53:19.785766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 23 19:53:19.785766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:19.794270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 255/505] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Oct 23 19:53:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:19.805075 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe3b5f-415f-4b62-aa92-41890e919a40 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:19.985361 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:19.991676 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0321a2d-aaac-4e59-883d-c2e7669ec7d7 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:20.013639 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fe3b5f-415f-4b62-aa92-41890e919a40 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:20.062457 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb64772-b559-47a1-8a8c-347a53509380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:20.063076 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb64772-b559-47a1-8a8c-347a53509380 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:20.063361 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb64772-b559-47a1-8a8c-347a53509380 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:20.063946 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb64772-b559-47a1-8a8c-347a53509380 None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:20.064631 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb64772-b559-47a1-8a8c-347a53509380 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:20.064936 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb64772-b559-47a1-8a8c-347a53509380 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:20.066390 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb64772-b559-47a1-8a8c-347a53509380 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:20.067411 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9b910-2fea-426c-b365-8223006121f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:20.068196 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9b910-2fea-426c-b365-8223006121f0 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:20.068547 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9b910-2fea-426c-b365-8223006121f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:20.068975 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9b910-2fea-426c-b365-8223006121f0 None admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:20.069749 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c9b910-2fea-426c-b365-8223006121f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:20.070126 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c9b910-2fea-426c-b365-8223006121f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:20.071659 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c9b910-2fea-426c-b365-8223006121f0 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:20.084787 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 251/506] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Oct 23 19:53:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:20.086114 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c0305-3bfc-456d-99b4-505913d342bf None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:20.086522 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c0305-3bfc-456d-99b4-505913d342bf None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:20.086853 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c0305-3bfc-456d-99b4-505913d342bf None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:20.088378 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 252/508] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:20.092228 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 256/508] 10.222.0.13 () {56 vars in 1135 bytes} [Sun Oct 23 19:53:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:53:20.099268 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb64772-b559-47a1-8a8c-347a53509380 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:20.099589 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043d1c1-d725-4f18-b07a-7634a0887ec3 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:20.099878 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043d1c1-d725-4f18-b07a-7634a0887ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:20.100277 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a043d1c1-d725-4f18-b07a-7634a0887ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:20.123358 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-a043d1c1-d725-4f18-b07a-7634a0887ec3 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:20.124610 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-a043d1c1-d725-4f18-b07a-7634a0887ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:20.164998 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:20.219318 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 253/509] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:53:20.220435 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043d1c1-d725-4f18-b07a-7634a0887ec3 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:20.230434 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb64772-b559-47a1-8a8c-347a53509380 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:20.290413 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c90088-5a70-4cc7-a8aa-16c6ec6a1f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:20.291453 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c90088-5a70-4cc7-a8aa-16c6ec6a1f79 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:20.292076 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c90088-5a70-4cc7-a8aa-16c6ec6a1f79 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:20.292411 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c90088-5a70-4cc7-a8aa-16c6ec6a1f79 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:20.355326 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:20.371835 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a043d1c1-d725-4f18-b07a-7634a0887ec3 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:20.376957 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c90088-5a70-4cc7-a8aa-16c6ec6a1f79 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:20.378456 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c90088-5a70-4cc7-a8aa-16c6ec6a1f79 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:20.416446 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39167f10-60d8-49d6-9bf3-d408eed12b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:20.417120 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39167f10-60d8-49d6-9bf3-d408eed12b8a None admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:20.417400 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39167f10-60d8-49d6-9bf3-d408eed12b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:20.417674 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39167f10-60d8-49d6-9bf3-d408eed12b8a None admin] PATH_INFO: `/v3/projects/d40947427f5f43e7b6c82e3605737767/users/0b3d512304254ffcaa8dad90abf1f9c1/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:20.419631 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39167f10-60d8-49d6-9bf3-d408eed12b8a None admin] RBAC: Authorizing `identity:create_grant(project_id=d40947427f5f43e7b6c82e3605737767, user_id=0b3d512304254ffcaa8dad90abf1f9c1, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:20.421061 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39167f10-60d8-49d6-9bf3-d408eed12b8a None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:20.435114 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 254/510] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Oct 23 19:53:20 2022] PUT /identity/v3/projects/d40947427f5f43e7b6c82e3605737767/users/0b3d512304254ffcaa8dad90abf1f9c1/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 19:53:20.436417 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc8d30-02ab-46be-a53c-719aa59c93fd None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:20.436795 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc8d30-02ab-46be-a53c-719aa59c93fd None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:20.437667 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fc8d30-02ab-46be-a53c-719aa59c93fd None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:20.444114 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 255/511] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:20.444114 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23cd12-4b13-433a-b3c7-7abd2ad2a87b None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:20.444114 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23cd12-4b13-433a-b3c7-7abd2ad2a87b None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:20.444114 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d23cd12-4b13-433a-b3c7-7abd2ad2a87b None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:20.445058 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:20.446467 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 256/512] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:20.448174 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022be943-130a-4787-8def-9ec0649dcf12 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:20.448458 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022be943-130a-4787-8def-9ec0649dcf12 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:20.448727 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022be943-130a-4787-8def-9ec0649dcf12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:20.469666 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-022be943-130a-4787-8def-9ec0649dcf12 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:20.471004 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-022be943-130a-4787-8def-9ec0649dcf12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:20.516774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 257/513] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Oct 23 19:53:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 19:53:20.518615 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407cb33f-c78b-4b80-8401-fcaff5246768 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:20.519207 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407cb33f-c78b-4b80-8401-fcaff5246768 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:20.519748 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407cb33f-c78b-4b80-8401-fcaff5246768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:20.539165 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-407cb33f-c78b-4b80-8401-fcaff5246768 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:20.540406 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-407cb33f-c78b-4b80-8401-fcaff5246768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:20.550946 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 257/514] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:53:20.550946 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022be943-130a-4787-8def-9ec0649dcf12 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:20.622427 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 258/515] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:53:20.623825 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407cb33f-c78b-4b80-8401-fcaff5246768 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:20.647659 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:20.673864 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022be943-130a-4787-8def-9ec0649dcf12 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:20.714856 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448370f-bd32-4eb3-98ad-d3d390f1eac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:20.715489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448370f-bd32-4eb3-98ad-d3d390f1eac6 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:20.715768 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448370f-bd32-4eb3-98ad-d3d390f1eac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:20.716045 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4448370f-bd32-4eb3-98ad-d3d390f1eac6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:20.728156 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4448370f-bd32-4eb3-98ad-d3d390f1eac6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:20.729929 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4448370f-bd32-4eb3-98ad-d3d390f1eac6 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:20.739929 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:20.748170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-4448370f-bd32-4eb3-98ad-d3d390f1eac6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:20.748170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:20.748170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:20.748170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:20.748170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:20.748170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:20.748170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:20.748170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:20.748170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:20.748170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:20.748170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:20.748170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:20.748170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:20.748170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:20.748170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:20.748170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:20.749169 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:20.749169 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:20.749169 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:20.749169 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:20.749169 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:20.749169 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:20.749169 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:20.749169 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:20.749169 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:20.749169 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:20.749169 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:20.749169 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:20.749169 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:20.749169 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:20.749169 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:20.749169 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:20.749169 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:20.750096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:20.750096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:20.750096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:20.750096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 258/516] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:20.755005 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448370f-bd32-4eb3-98ad-d3d390f1eac6 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:20.776854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407cb33f-c78b-4b80-8401-fcaff5246768 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:20.839325 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e4089-69c4-404f-907a-5da6169a547c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:20.840265 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e4089-69c4-404f-907a-5da6169a547c None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:20.840543 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e4089-69c4-404f-907a-5da6169a547c None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:20.840812 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98e4089-69c4-404f-907a-5da6169a547c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:20.860422 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e4089-69c4-404f-907a-5da6169a547c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:20.861883 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98e4089-69c4-404f-907a-5da6169a547c None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:20.879900 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-d98e4089-69c4-404f-907a-5da6169a547c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:20.879900 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:20.879900 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:20.879900 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:20.879900 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:20.879900 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:20.879900 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:20.879900 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:20.879900 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:20.879900 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:20.879900 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:20.879900 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:20.879900 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:20.879900 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:20.879900 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:20.879900 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:20.880883 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:20.880883 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:20.880883 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:20.880883 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:20.880883 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:20.880883 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:20.880883 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:20.880883 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:20.880883 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:20.880883 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:20.880883 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:20.880883 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:20.880883 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:20.880883 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:20.880883 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:20.880883 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:20.880883 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:20.881861 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:20.881861 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:20.881861 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:20.881861 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 259/517] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:20.881861 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e4089-69c4-404f-907a-5da6169a547c None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:20.901653 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4448370f-bd32-4eb3-98ad-d3d390f1eac6 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:20.933406 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:20.940857 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1db747b-9def-44fd-971e-3f0afbdf5d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:20.941809 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db747b-9def-44fd-971e-3f0afbdf5d94 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:20.942343 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db747b-9def-44fd-971e-3f0afbdf5d94 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:20.942822 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db747b-9def-44fd-971e-3f0afbdf5d94 None admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:20.943742 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1db747b-9def-44fd-971e-3f0afbdf5d94 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:20.944387 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1db747b-9def-44fd-971e-3f0afbdf5d94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:20.946137 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1db747b-9def-44fd-971e-3f0afbdf5d94 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:20.958871 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 259/518] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:53:20.962293 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1db747b-9def-44fd-971e-3f0afbdf5d94 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:21.011481 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98e4089-69c4-404f-907a-5da6169a547c None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:21.042108 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:21.048059 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533170be-7fe2-469f-96d6-4ab39544410c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:21.048704 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533170be-7fe2-469f-96d6-4ab39544410c None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:21.048982 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533170be-7fe2-469f-96d6-4ab39544410c None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:21.049259 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533170be-7fe2-469f-96d6-4ab39544410c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:21.049847 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533170be-7fe2-469f-96d6-4ab39544410c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:21.051238 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533170be-7fe2-469f-96d6-4ab39544410c None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:21.064023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-533170be-7fe2-469f-96d6-4ab39544410c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 23 19:53:21.064023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:21.064023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 19:53:21.064023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 19:53:21.064023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 19:53:21.064023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 19:53:21.064023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 19:53:21.064023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:21.064023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 23 19:53:21.064023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:21.064023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 19:53:21.064023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:21.064023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:21.064023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:21.064023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:21.064023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:21.064023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:21.065039 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:21.065039 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:21.065039 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:21.065039 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:21.065039 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:21.065039 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:21.065039 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 19:53:21.065039 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 19:53:21.065039 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 19:53:21.065039 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 19:53:21.065039 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:21.065039 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:21.065039 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:21.065039 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:21.065039 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:21.065039 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:21.065869 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:21.065869 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:21.065869 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:21.065869 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:21.065869 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:21.065869 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:21.065869 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:21.065869 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:21.065869 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:21.065869 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:21.065869 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 19:53:21.065869 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:21.065869 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 23 19:53:21.065869 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:21.070504 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 260/519] 10.222.0.13 () {56 vars in 1129 bytes} [Sun Oct 23 19:53:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:21.071342 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533170be-7fe2-469f-96d6-4ab39544410c None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:21.074151 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1db747b-9def-44fd-971e-3f0afbdf5d94 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:21.112696 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a757fcb-db0d-4f51-8569-dfc667179f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:21.113551 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a757fcb-db0d-4f51-8569-dfc667179f16 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:21.113982 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a757fcb-db0d-4f51-8569-dfc667179f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:21.114854 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a757fcb-db0d-4f51-8569-dfc667179f16 None admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:21.115659 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a757fcb-db0d-4f51-8569-dfc667179f16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:21.116132 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a757fcb-db0d-4f51-8569-dfc667179f16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:21.118323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a757fcb-db0d-4f51-8569-dfc667179f16 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:21.130664 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 260/520] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:21.131478 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a757fcb-db0d-4f51-8569-dfc667179f16 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:21.180060 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533170be-7fe2-469f-96d6-4ab39544410c None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:21.184713 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:21.215700 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b296d1-020c-448f-a332-f01ea470c002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:21.216345 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b296d1-020c-448f-a332-f01ea470c002 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:21.216603 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b296d1-020c-448f-a332-f01ea470c002 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:21.216879 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b296d1-020c-448f-a332-f01ea470c002 None admin] PATH_INFO: `/v3/services/network` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:21.217460 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b296d1-020c-448f-a332-f01ea470c002 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:21.218936 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b296d1-020c-448f-a332-f01ea470c002 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:21.227946 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-23b296d1-020c-448f-a332-f01ea470c002 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 23 19:53:21.227946 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:21.227946 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 19:53:21.227946 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 19:53:21.227946 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 19:53:21.227946 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 19:53:21.227946 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 19:53:21.227946 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:21.227946 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 23 19:53:21.227946 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:21.227946 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 19:53:21.227946 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:21.227946 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:21.227946 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:21.227946 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:21.227946 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:21.227946 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:21.228922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:21.228922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:21.228922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:21.228922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:21.228922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:21.228922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:21.228922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 19:53:21.228922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 19:53:21.228922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 19:53:21.228922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 19:53:21.228922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:21.228922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:21.228922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:21.228922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:21.228922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:21.228922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:21.229758 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:21.229758 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:21.229758 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:21.229758 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:21.229758 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:21.229758 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:21.229758 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:21.229758 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:21.229758 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:21.229758 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:21.229758 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 19:53:21.229758 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:21.229758 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 23 19:53:21.229758 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:21.229758 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 261/521] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:53:21 2022] GET /identity/v3/services/network => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:21.231062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b296d1-020c-448f-a332-f01ea470c002 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:21.245047 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a757fcb-db0d-4f51-8569-dfc667179f16 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:21.281065 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef703361-8630-4e6a-87ab-b4fc686bd8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:21.282078 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef703361-8630-4e6a-87ab-b4fc686bd8eb None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:21.282656 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef703361-8630-4e6a-87ab-b4fc686bd8eb None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:21.282947 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef703361-8630-4e6a-87ab-b4fc686bd8eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:21.291985 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef703361-8630-4e6a-87ab-b4fc686bd8eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:21.293703 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:21.293902 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef703361-8630-4e6a-87ab-b4fc686bd8eb None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:21.307365 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-ef703361-8630-4e6a-87ab-b4fc686bd8eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:21.307365 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:21.307365 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:21.307365 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:21.307365 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:21.307365 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:21.307365 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:21.307365 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:21.307365 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:21.307365 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:21.307365 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:21.307365 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:21.307365 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 19:53:21.307365 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 19:53:21.307365 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 19:53:21.308312 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 19:53:21.308312 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:21.308312 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:21.308312 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:21.308312 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:21.308312 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:21.308312 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:21.308312 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:21.308312 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:21.308312 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:21.308312 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:21.308312 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:21.308312 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:21.308312 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:21.308312 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:21.308312 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:21.308312 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:21.309222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 19:53:21.309222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 19:53:21.309222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 19:53:21.309222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 19:53:21.309222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 19:53:21.309222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 19:53:21.309222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:21.309222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:21.313773 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 261/522] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:53:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:21.314225 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef703361-8630-4e6a-87ab-b4fc686bd8eb None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:21.337383 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b296d1-020c-448f-a332-f01ea470c002 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:21.371208 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002f3cb-ad04-4886-8cfc-87c218ad08c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:21.371748 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002f3cb-ad04-4886-8cfc-87c218ad08c7 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:21.372023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002f3cb-ad04-4886-8cfc-87c218ad08c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:21.372291 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2002f3cb-ad04-4886-8cfc-87c218ad08c7 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:21.373035 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002f3cb-ad04-4886-8cfc-87c218ad08c7 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:21.373332 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002f3cb-ad04-4886-8cfc-87c218ad08c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:21.374768 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2002f3cb-ad04-4886-8cfc-87c218ad08c7 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:21.380260 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 262/523] 10.222.0.13 () {56 vars in 1124 bytes} [Sun Oct 23 19:53:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:21.381022 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002f3cb-ad04-4886-8cfc-87c218ad08c7 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:21.432481 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef703361-8630-4e6a-87ab-b4fc686bd8eb None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:21.443725 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:21.470479 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22cafb-c66d-40fd-8d27-5e1ec6a74b3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:21.471345 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22cafb-c66d-40fd-8d27-5e1ec6a74b3e None admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:21.471642 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22cafb-c66d-40fd-8d27-5e1ec6a74b3e None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:21.471917 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd22cafb-c66d-40fd-8d27-5e1ec6a74b3e None admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:21.472573 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd22cafb-c66d-40fd-8d27-5e1ec6a74b3e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:21.473996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd22cafb-c66d-40fd-8d27-5e1ec6a74b3e None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:21.493111 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2002f3cb-ad04-4886-8cfc-87c218ad08c7 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:21.496141 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 262/524] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Oct 23 19:53:21 2022] POST /identity/v3/users => generated 254 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:53:21.496568 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22cafb-c66d-40fd-8d27-5e1ec6a74b3e None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:21.541171 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:21.559002 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0243b630-2aa7-4de5-b58c-71ac7628472b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:21.560343 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243b630-2aa7-4de5-b58c-71ac7628472b None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:21.560849 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243b630-2aa7-4de5-b58c-71ac7628472b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:21.561623 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0243b630-2aa7-4de5-b58c-71ac7628472b None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:21.563923 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0243b630-2aa7-4de5-b58c-71ac7628472b None admin] RBAC: Adding query filter params (name=network) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:21.564742 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0243b630-2aa7-4de5-b58c-71ac7628472b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:21.566918 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0243b630-2aa7-4de5-b58c-71ac7628472b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:21.574734 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 263/525] 10.222.0.13 () {56 vars in 1112 bytes} [Sun Oct 23 19:53:21 2022] GET /identity/v3/services?name=network => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:21.575959 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0243b630-2aa7-4de5-b58c-71ac7628472b None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:21.621639 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd22cafb-c66d-40fd-8d27-5e1ec6a74b3e None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:21.673386 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28acab-a246-4036-89f7-2f25aa9a29d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:21.674359 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28acab-a246-4036-89f7-2f25aa9a29d0 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:21.675248 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28acab-a246-4036-89f7-2f25aa9a29d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:21.675703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28acab-a246-4036-89f7-2f25aa9a29d0 None admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:21.676591 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d28acab-a246-4036-89f7-2f25aa9a29d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:21.677089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d28acab-a246-4036-89f7-2f25aa9a29d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:21.679141 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d28acab-a246-4036-89f7-2f25aa9a29d0 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:21.695935 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 263/526] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Oct 23 19:53:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:21.696910 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28acab-a246-4036-89f7-2f25aa9a29d0 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:21.712915 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0243b630-2aa7-4de5-b58c-71ac7628472b None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:21.727825 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:21.757750 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be5f839-2db1-4748-9917-2823b619cb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:21.759358 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5f839-2db1-4748-9917-2823b619cb11 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:21.759790 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5f839-2db1-4748-9917-2823b619cb11 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:21.760233 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be5f839-2db1-4748-9917-2823b619cb11 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:21.761072 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be5f839-2db1-4748-9917-2823b619cb11 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:21.761524 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be5f839-2db1-4748-9917-2823b619cb11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:21.763959 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be5f839-2db1-4748-9917-2823b619cb11 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:21.778828 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 264/527] 10.222.0.13 () {56 vars in 1124 bytes} [Sun Oct 23 19:53:21 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:21.778828 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be5f839-2db1-4748-9917-2823b619cb11 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:21.850234 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28acab-a246-4036-89f7-2f25aa9a29d0 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:21.901308 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e60168-02bf-46fd-8354-8c2c4f87b7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:21.902569 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e60168-02bf-46fd-8354-8c2c4f87b7fe None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:21.903086 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e60168-02bf-46fd-8354-8c2c4f87b7fe None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:21.903560 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e60168-02bf-46fd-8354-8c2c4f87b7fe None admin] PATH_INFO: `/v3/services` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:21.904415 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e60168-02bf-46fd-8354-8c2c4f87b7fe None admin] RBAC: Adding query filter params (type=network) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:21.904944 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e60168-02bf-46fd-8354-8c2c4f87b7fe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:21.906619 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e60168-02bf-46fd-8354-8c2c4f87b7fe None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:21.913215 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:21.915199 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 264/528] 10.222.0.13 () {56 vars in 1112 bytes} [Sun Oct 23 19:53:21 2022] GET /identity/v3/services?type=network => generated 124 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:21.980354 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be5f839-2db1-4748-9917-2823b619cb11 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:22.019840 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135b5b6-bb31-49c1-9966-aca10296dcb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:22.020477 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135b5b6-bb31-49c1-9966-aca10296dcb3 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:22.020759 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135b5b6-bb31-49c1-9966-aca10296dcb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:22.021031 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135b5b6-bb31-49c1-9966-aca10296dcb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:22.021794 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135b5b6-bb31-49c1-9966-aca10296dcb3 None admin] RBAC: Adding query filter params (role.id=5e0524ede3ee479c923c7b609a0ca402, scope.project.id=d40947427f5f43e7b6c82e3605737767, user.id=15f603d1e7f54751aa79317e2ea00a31) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:22.022109 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135b5b6-bb31-49c1-9966-aca10296dcb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:22.023750 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135b5b6-bb31-49c1-9966-aca10296dcb3 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:22.031255 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 265/529] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Oct 23 19:53:21 2022] GET /identity/v3/role_assignments?user.id=15f603d1e7f54751aa79317e2ea00a31&scope.project.id=d40947427f5f43e7b6c82e3605737767&role.id=5e0524ede3ee479c923c7b609a0ca402 => generated 608 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:22.829937 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674cced-9057-4987-b965-a3eadd7289f1 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:22.830491 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674cced-9057-4987-b965-a3eadd7289f1 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:22.830491 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e674cced-9057-4987-b965-a3eadd7289f1 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:22.832124 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 265/530] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:22.837623 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1cd8a-f9ee-4b51-83a3-7dab4654357f None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:22.838035 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1cd8a-f9ee-4b51-83a3-7dab4654357f None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:22.838205 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d1cd8a-f9ee-4b51-83a3-7dab4654357f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:22.857708 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-75d1cd8a-f9ee-4b51-83a3-7dab4654357f None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:22.859486 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-75d1cd8a-f9ee-4b51-83a3-7dab4654357f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:22.946114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 266/531] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:53:23.330626 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674cced-9057-4987-b965-a3eadd7289f1 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:23.451394 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:23.470352 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e674cced-9057-4987-b965-a3eadd7289f1 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:23.515790 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae166bf-1406-45b4-b562-c15a89046a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:23.516421 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae166bf-1406-45b4-b562-c15a89046a2b None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:23.516702 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae166bf-1406-45b4-b562-c15a89046a2b None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:23.516975 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae166bf-1406-45b4-b562-c15a89046a2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:23.531541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae166bf-1406-45b4-b562-c15a89046a2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:53:23.536391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-9ae166bf-1406-45b4-b562-c15a89046a2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:53:23.536391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:23.536391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:23.536391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:23.536391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:23.536391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:23.536391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:23.536391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:23.536391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:23.536391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:23.536391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:23.536391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:23.536391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:53:23.536391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:53:23.536391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:53:23.536391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:53:23.537320 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:53:23.537320 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:53:23.537320 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:53:23.537320 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:53:23.537320 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:23.537320 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:23.537320 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:23.537320 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:23.537320 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:23.537320 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:23.537320 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:23.537320 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:23.537320 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:23.537320 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:23.537320 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:23.537320 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:23.538162 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:23.538162 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:23.538162 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:23.538162 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:23.538162 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:53:23.538162 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:53:23.538162 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:53:23.538162 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:53:23.538162 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:53:23.538162 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:53:23.538162 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:53:23.538162 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:23.538162 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 266/532] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Oct 23 19:53:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:23.543493 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d1cd8a-f9ee-4b51-83a3-7dab4654357f None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:23.614280 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:23.687535 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d1cd8a-f9ee-4b51-83a3-7dab4654357f None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:23.739830 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99d459-f6a8-4a33-8b87-506fb1134ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:23.740451 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99d459-f6a8-4a33-8b87-506fb1134ff4 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:23.740727 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99d459-f6a8-4a33-8b87-506fb1134ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:23.740994 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e99d459-f6a8-4a33-8b87-506fb1134ff4 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:23.741652 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99d459-f6a8-4a33-8b87-506fb1134ff4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:23.741950 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99d459-f6a8-4a33-8b87-506fb1134ff4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:23.743340 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e99d459-f6a8-4a33-8b87-506fb1134ff4 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:23.754391 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 267/533] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:53:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:23.760124 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae166bf-1406-45b4-b562-c15a89046a2b None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:23.904339 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae166bf-1406-45b4-b562-c15a89046a2b None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:23.952993 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7216bb-5219-4bdf-bb43-f11ab82f34af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:23.954011 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7216bb-5219-4bdf-bb43-f11ab82f34af None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:23.954512 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7216bb-5219-4bdf-bb43-f11ab82f34af None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:23.955007 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7216bb-5219-4bdf-bb43-f11ab82f34af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:23.976155 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7216bb-5219-4bdf-bb43-f11ab82f34af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:23.977876 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7216bb-5219-4bdf-bb43-f11ab82f34af None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:23.998175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-5f7216bb-5219-4bdf-bb43-f11ab82f34af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:23.998175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:23.998175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:23.998175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:23.998175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:23.998175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:23.998175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:23.998175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:23.998175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:23.998175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:23.998175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:23.998175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:23.998175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:23.998175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:23.998175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:23.998175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:23.999168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:23.999168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:23.999168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:23.999168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:23.999168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:23.999168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:23.999168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:23.999168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:23.999168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:23.999168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:23.999168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:23.999168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:23.999168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:23.999168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:23.999168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:23.999168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:23.999168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:24.000584 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:24.000584 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:24.000584 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:24.002383 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 267/534] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:24.008300 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99d459-f6a8-4a33-8b87-506fb1134ff4 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:24.184832 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e99d459-f6a8-4a33-8b87-506fb1134ff4 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:24.200140 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:24.235747 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2bccd0-1bb8-4ce2-a4dc-aac6ce9e5618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:24.236463 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2bccd0-1bb8-4ce2-a4dc-aac6ce9e5618 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:24.236745 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2bccd0-1bb8-4ce2-a4dc-aac6ce9e5618 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:24.237067 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2bccd0-1bb8-4ce2-a4dc-aac6ce9e5618 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:24.237715 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2bccd0-1bb8-4ce2-a4dc-aac6ce9e5618 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:24.238012 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2bccd0-1bb8-4ce2-a4dc-aac6ce9e5618 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:24.239425 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2bccd0-1bb8-4ce2-a4dc-aac6ce9e5618 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:24.257165 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 268/535] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:24.262761 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7216bb-5219-4bdf-bb43-f11ab82f34af None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:24.268620 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:24.318813 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df78949-4d9d-4c1b-b56c-c1cd8b605ede None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:24.319141 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df78949-4d9d-4c1b-b56c-c1cd8b605ede None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:24.319359 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df78949-4d9d-4c1b-b56c-c1cd8b605ede None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:24.321104 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 269/536] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:24.326799 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743dba80-8076-4bbc-996f-0e0f87720d62 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:24.327082 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743dba80-8076-4bbc-996f-0e0f87720d62 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:24.327880 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743dba80-8076-4bbc-996f-0e0f87720d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:24.354785 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-743dba80-8076-4bbc-996f-0e0f87720d62 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:24.356324 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-743dba80-8076-4bbc-996f-0e0f87720d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:24.407319 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7216bb-5219-4bdf-bb43-f11ab82f34af None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:24.430762 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 270/537] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:53:24.432460 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27343bea-721a-44c1-9015-0e3c3b1640ba None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:24.432947 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27343bea-721a-44c1-9015-0e3c3b1640ba None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:24.433417 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27343bea-721a-44c1-9015-0e3c3b1640ba None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:24.435366 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 271/538] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:24.441899 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba41442-68e5-4965-8443-543d55592b5b None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:24.442609 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba41442-68e5-4965-8443-543d55592b5b None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:24.443186 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba41442-68e5-4965-8443-543d55592b5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:24.443715 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3727a16c-e984-4260-abc4-c55ce3830c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:24.444339 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3727a16c-e984-4260-abc4-c55ce3830c9a None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:24.444616 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3727a16c-e984-4260-abc4-c55ce3830c9a None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:24.444883 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3727a16c-e984-4260-abc4-c55ce3830c9a None admin] PATH_INFO: `/v3/users/nova` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:24.452898 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3727a16c-e984-4260-abc4-c55ce3830c9a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:24.454354 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3727a16c-e984-4260-abc4-c55ce3830c9a None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:24.463644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-cba41442-68e5-4965-8443-543d55592b5b None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:24.465070 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-cba41442-68e5-4965-8443-543d55592b5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:24.466313 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-3727a16c-e984-4260-abc4-c55ce3830c9a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 23 19:53:24.466313 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:24.466313 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:24.466313 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:24.466313 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:24.466313 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:24.466313 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:24.466313 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:24.466313 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:24.466313 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:24.466313 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:24.466313 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:24.466313 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 19:53:24.466313 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 19:53:24.466313 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 19:53:24.466313 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 19:53:24.467234 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:24.467234 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:24.467234 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 19:53:24.467234 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:24.467234 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 19:53:24.467234 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:24.467234 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:24.467234 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:24.467234 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:24.467234 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:24.467234 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:24.467234 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:24.467234 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:24.467234 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:24.467234 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:24.467234 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:24.467234 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:24.467234 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:24.468693 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:24.468693 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:24.468693 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 19:53:24.468693 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 19:53:24.468693 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 19:53:24.468693 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 19:53:24.468693 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 19:53:24.468693 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 19:53:24.468693 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 23 19:53:24.468693 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:24.472635 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 268/539] 10.222.0.13 () {56 vars in 1093 bytes} [Sun Oct 23 19:53:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:24.473980 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3727a16c-e984-4260-abc4-c55ce3830c9a None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:24.542296 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:24.543977 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 272/540] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:53:24.612975 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3727a16c-e984-4260-abc4-c55ce3830c9a None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:24.661485 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87790397-680e-4f55-a523-2e9a1de5d01a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:24.662109 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87790397-680e-4f55-a523-2e9a1de5d01a None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:24.662415 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87790397-680e-4f55-a523-2e9a1de5d01a None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:24.662686 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87790397-680e-4f55-a523-2e9a1de5d01a None admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:24.663425 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87790397-680e-4f55-a523-2e9a1de5d01a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:24.663728 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87790397-680e-4f55-a523-2e9a1de5d01a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:24.665164 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87790397-680e-4f55-a523-2e9a1de5d01a None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:24.687293 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 269/541] 10.222.0.13 () {56 vars in 1133 bytes} [Sun Oct 23 19:53:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:24.693094 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba41442-68e5-4965-8443-543d55592b5b None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:24.723398 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87790397-680e-4f55-a523-2e9a1de5d01a None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:24.733238 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:24.811719 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba41442-68e5-4965-8443-543d55592b5b None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:24.847092 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbaac92-10f7-4ec7-95d3-cd2e7b98019e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:24.847710 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbaac92-10f7-4ec7-95d3-cd2e7b98019e None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:24.847988 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbaac92-10f7-4ec7-95d3-cd2e7b98019e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:24.848260 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbaac92-10f7-4ec7-95d3-cd2e7b98019e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:24.866116 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87790397-680e-4f55-a523-2e9a1de5d01a None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:24.886893 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:24.902553 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661e104a-4c41-4e64-b12e-8421f0698706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:24.903184 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e104a-4c41-4e64-b12e-8421f0698706 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:24.903460 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e104a-4c41-4e64-b12e-8421f0698706 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:24.903731 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661e104a-4c41-4e64-b12e-8421f0698706 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:24.915586 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661e104a-4c41-4e64-b12e-8421f0698706 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:53:24.920852 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-661e104a-4c41-4e64-b12e-8421f0698706 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 19:53:24.920852 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:24.920852 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:24.920852 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:24.920852 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:24.920852 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:24.920852 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:24.920852 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:24.920852 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:24.920852 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:24.920852 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:24.920852 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:24.920852 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:53:24.920852 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:53:24.920852 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:53:24.920852 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:53:24.922066 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:53:24.922066 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:53:24.922066 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:53:24.922066 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:53:24.922066 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:24.922066 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:24.922066 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:24.922066 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:24.922066 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:24.922066 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:24.922066 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:24.922066 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:24.922066 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:24.922066 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:24.922066 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:24.922066 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:24.923328 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:24.923328 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:24.923328 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:24.923328 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:24.923328 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:53:24.923328 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:53:24.923328 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:53:24.923328 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:53:24.923328 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:53:24.923328 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:53:24.923328 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 19:53:24.923328 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:24.923328 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 270/542] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Oct 23 19:53:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:53:24.924179 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c09fba-e36d-49ab-8de3-42aed4c34136 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:24.924179 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c09fba-e36d-49ab-8de3-42aed4c34136 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:24.924179 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c09fba-e36d-49ab-8de3-42aed4c34136 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:24.926202 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 271/543] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:24.926991 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c09fba-e36d-49ab-8de3-42aed4c34136 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:24.932772 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbaac92-10f7-4ec7-95d3-cd2e7b98019e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:24.934283 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbaac92-10f7-4ec7-95d3-cd2e7b98019e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:24.964814 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:25.052736 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 273/544] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Oct 23 19:53:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 19:53:25.052736 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbaac92-10f7-4ec7-95d3-cd2e7b98019e None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:25.053418 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c09fba-e36d-49ab-8de3-42aed4c34136 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:25.084961 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d3c91f-0a78-4527-9f56-c7bc1708ae3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:25.085565 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3c91f-0a78-4527-9f56-c7bc1708ae3c None admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:25.085841 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3c91f-0a78-4527-9f56-c7bc1708ae3c None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:25.086112 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d3c91f-0a78-4527-9f56-c7bc1708ae3c None admin] PATH_INFO: `/v3/services` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:25.086817 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d3c91f-0a78-4527-9f56-c7bc1708ae3c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:25.088165 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d3c91f-0a78-4527-9f56-c7bc1708ae3c None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:25.103737 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 272/545] 10.222.0.13 () {60 vars in 1141 bytes} [Sun Oct 23 19:53:24 2022] POST /identity/v3/services => generated 251 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:53:25.107695 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06230c2-fd04-445b-9f7f-2818293916bc None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:25.107993 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06230c2-fd04-445b-9f7f-2818293916bc None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:25.108297 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06230c2-fd04-445b-9f7f-2818293916bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:25.130232 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-b06230c2-fd04-445b-9f7f-2818293916bc None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:25.131504 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-b06230c2-fd04-445b-9f7f-2818293916bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:25.158900 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbaac92-10f7-4ec7-95d3-cd2e7b98019e None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:25.171848 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:25.190471 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:25.211748 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f2c4a-ebf3-4ac1-a0de-ff3078a84c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:25.212373 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f2c4a-ebf3-4ac1-a0de-ff3078a84c35 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:25.212647 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f2c4a-ebf3-4ac1-a0de-ff3078a84c35 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:25.212916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f2c4a-ebf3-4ac1-a0de-ff3078a84c35 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:25.213585 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f2c4a-ebf3-4ac1-a0de-ff3078a84c35 None admin] RBAC: Adding query filter params (name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:25.213884 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f2c4a-ebf3-4ac1-a0de-ff3078a84c35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:25.215430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f2c4a-ebf3-4ac1-a0de-ff3078a84c35 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:25.217079 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 273/546] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:25 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:53:25.219022 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1a469-01a6-4b0d-bde2-be01fd8170f8 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:25.219201 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1a469-01a6-4b0d-bde2-be01fd8170f8 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:25.219437 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a1a469-01a6-4b0d-bde2-be01fd8170f8 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:25.224895 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 274/547] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:25.224895 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1a469-01a6-4b0d-bde2-be01fd8170f8 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:25.233862 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 274/548] 10.222.0.13 () {56 vars in 1103 bytes} [Sun Oct 23 19:53:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:25.235603 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02c07a-7b99-4d24-b98f-7b92c66b8bfe None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:25.235979 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02c07a-7b99-4d24-b98f-7b92c66b8bfe None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:25.236818 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c02c07a-7b99-4d24-b98f-7b92c66b8bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:25.256695 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-2c02c07a-7b99-4d24-b98f-7b92c66b8bfe None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:25.258289 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2c02c07a-7b99-4d24-b98f-7b92c66b8bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:25.341997 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 275/549] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:25 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 19:53:25.346331 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02c07a-7b99-4d24-b98f-7b92c66b8bfe None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:25.375324 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a1a469-01a6-4b0d-bde2-be01fd8170f8 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:25.437449 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e3c5c7-ad6f-4564-ad4f-35b5929ac75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:25.438083 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3c5c7-ad6f-4564-ad4f-35b5929ac75c None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:25.438391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3c5c7-ad6f-4564-ad4f-35b5929ac75c None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:25.438919 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e3c5c7-ad6f-4564-ad4f-35b5929ac75c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:25.455734 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e3c5c7-ad6f-4564-ad4f-35b5929ac75c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:25.457170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e3c5c7-ad6f-4564-ad4f-35b5929ac75c None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:25.485552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-d5e3c5c7-ad6f-4564-ad4f-35b5929ac75c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:25.485552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:25.485552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:25.485552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:25.485552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:25.485552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:25.485552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:25.485552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:25.485552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:25.485552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:25.485552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:25.485552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:25.485552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:25.485552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:25.485552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:25.485552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:25.486554 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:25.486554 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:25.486554 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:25.486554 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:25.486554 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:25.486554 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:25.486554 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:25.486554 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:25.486554 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:25.486554 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:25.486554 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:25.486554 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:25.486554 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:25.486554 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:25.486554 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:25.486554 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:25.486554 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:25.487462 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:25.487462 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:25.487462 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:25.488870 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 275/550] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:25.494897 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e3c5c7-ad6f-4564-ad4f-35b5929ac75c None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:25.508588 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c02c07a-7b99-4d24-b98f-7b92c66b8bfe None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:25.522952 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:25.529178 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:25.550006 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69e74c-3547-4896-ab02-6e2c1f1bd64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:25.550747 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69e74c-3547-4896-ab02-6e2c1f1bd64b None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:25.551026 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69e74c-3547-4896-ab02-6e2c1f1bd64b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:25.551297 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f69e74c-3547-4896-ab02-6e2c1f1bd64b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:25.562676 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69e74c-3547-4896-ab02-6e2c1f1bd64b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:25.564414 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f69e74c-3547-4896-ab02-6e2c1f1bd64b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:25.583703 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-5f69e74c-3547-4896-ab02-6e2c1f1bd64b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:25.583703 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:25.583703 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:25.583703 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:25.583703 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:25.583703 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:25.583703 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:25.583703 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:25.583703 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:25.583703 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:25.583703 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:25.583703 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:25.583703 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:25.583703 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:25.583703 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:25.583703 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:25.584663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:25.584663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:25.584663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:25.584663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:25.584663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:25.584663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:25.584663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:25.584663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:25.584663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:25.584663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:25.584663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:25.584663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:25.584663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:25.584663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:25.584663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:25.584663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:25.584663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:25.586135 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:25.586135 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:25.586135 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:25.586135 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 276/551] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:25.586135 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69e74c-3547-4896-ab02-6e2c1f1bd64b None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:25.638578 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e3c5c7-ad6f-4564-ad4f-35b5929ac75c None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:25.677849 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5029a1a4-1c3e-4174-bb58-266892dcd285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:25.678652 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029a1a4-1c3e-4174-bb58-266892dcd285 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:25.678934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029a1a4-1c3e-4174-bb58-266892dcd285 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:25.679500 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5029a1a4-1c3e-4174-bb58-266892dcd285 None admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:25.680151 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5029a1a4-1c3e-4174-bb58-266892dcd285 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:25.680456 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5029a1a4-1c3e-4174-bb58-266892dcd285 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:25.682267 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5029a1a4-1c3e-4174-bb58-266892dcd285 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:25.695890 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f69e74c-3547-4896-ab02-6e2c1f1bd64b None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:25.701863 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 276/552] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:25.701863 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5029a1a4-1c3e-4174-bb58-266892dcd285 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:25.738268 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b7dc03-e9ad-4711-88ee-37940e85a360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:25.738828 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7dc03-e9ad-4711-88ee-37940e85a360 None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:25.739441 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7dc03-e9ad-4711-88ee-37940e85a360 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:25.739551 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7dc03-e9ad-4711-88ee-37940e85a360 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:25.740121 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b7dc03-e9ad-4711-88ee-37940e85a360 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:25.741620 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b7dc03-e9ad-4711-88ee-37940e85a360 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:25.762396 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 277/553] 10.222.0.13 () {60 vars in 1141 bytes} [Sun Oct 23 19:53:25 2022] POST /identity/v3/services => generated 240 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:53:25.763605 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b7dc03-e9ad-4711-88ee-37940e85a360 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:25.793287 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:25.803178 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:25.825266 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5029a1a4-1c3e-4174-bb58-266892dcd285 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:25.882279 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47bde4b-e282-4631-aaed-19b6c6b08e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:25.883408 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47bde4b-e282-4631-aaed-19b6c6b08e5d None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:25.883903 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47bde4b-e282-4631-aaed-19b6c6b08e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:25.884382 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47bde4b-e282-4631-aaed-19b6c6b08e5d None admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:25.885426 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47bde4b-e282-4631-aaed-19b6c6b08e5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:25.885942 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47bde4b-e282-4631-aaed-19b6c6b08e5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:25.887617 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47bde4b-e282-4631-aaed-19b6c6b08e5d None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:25.907881 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 277/554] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:25.907881 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47bde4b-e282-4631-aaed-19b6c6b08e5d None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:25.910441 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b7dc03-e9ad-4711-88ee-37940e85a360 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:25.959584 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a8f812-a565-49db-bdc1-aaed8fadc47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:25.960231 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a8f812-a565-49db-bdc1-aaed8fadc47c None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:25.960505 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a8f812-a565-49db-bdc1-aaed8fadc47c None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:25.961230 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a8f812-a565-49db-bdc1-aaed8fadc47c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:25.979025 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a8f812-a565-49db-bdc1-aaed8fadc47c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:53:25.986880 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-b7a8f812-a565-49db-bdc1-aaed8fadc47c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:53:25.986880 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:25.986880 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:25.986880 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:25.986880 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:25.986880 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:25.986880 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:25.986880 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:25.986880 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:25.986880 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:25.986880 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:25.986880 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:25.986880 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:53:25.986880 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:53:25.986880 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:53:25.986880 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:53:25.987849 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:53:25.987849 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:53:25.987849 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:53:25.987849 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:53:25.987849 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:25.987849 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:25.987849 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:25.987849 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:25.987849 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:25.987849 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:25.987849 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:25.987849 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:25.987849 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:25.987849 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:25.987849 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:25.987849 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:25.988752 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:25.988752 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:25.988752 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:25.988752 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:25.988752 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:53:25.988752 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:53:25.988752 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:53:25.988752 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:53:25.988752 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:53:25.988752 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:53:25.988752 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:53:25.988752 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:25.988752 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 278/555] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Oct 23 19:53:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:25.989475 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a8f812-a565-49db-bdc1-aaed8fadc47c None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:26.034317 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47bde4b-e282-4631-aaed-19b6c6b08e5d None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:26.070676 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e07d813-607e-46a5-93d0-d66e7549eaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:26.071677 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07d813-607e-46a5-93d0-d66e7549eaaa None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:26.072160 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07d813-607e-46a5-93d0-d66e7549eaaa None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:26.072628 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07d813-607e-46a5-93d0-d66e7549eaaa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:26.074849 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:26.081847 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e07d813-607e-46a5-93d0-d66e7549eaaa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:26.083600 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e07d813-607e-46a5-93d0-d66e7549eaaa None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:26.088874 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:26.096963 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-2e07d813-607e-46a5-93d0-d66e7549eaaa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:26.096963 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:26.096963 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:26.096963 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:26.096963 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:26.096963 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:26.096963 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:26.096963 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:26.096963 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:26.096963 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:26.096963 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:26.096963 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:26.096963 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 19:53:26.096963 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 19:53:26.096963 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 19:53:26.097892 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 19:53:26.097892 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:26.097892 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:26.097892 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:26.097892 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:26.097892 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:26.097892 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:26.097892 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:26.097892 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:26.097892 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:26.097892 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:26.097892 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:26.097892 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:26.097892 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:26.097892 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:26.097892 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:26.097892 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:26.098829 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 19:53:26.098829 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 19:53:26.098829 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 19:53:26.098829 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 19:53:26.098829 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 19:53:26.098829 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 19:53:26.098829 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:26.098829 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:26.101201 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 278/556] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:53:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:26.105861 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a8f812-a565-49db-bdc1-aaed8fadc47c None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:26.107205 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e07d813-607e-46a5-93d0-d66e7549eaaa None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:26.141559 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0521da0-f8b8-440a-b86b-fb26b66fca4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:26.142204 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0521da0-f8b8-440a-b86b-fb26b66fca4d None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:26.142800 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0521da0-f8b8-440a-b86b-fb26b66fca4d None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:26.142911 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0521da0-f8b8-440a-b86b-fb26b66fca4d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:26.151390 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0521da0-f8b8-440a-b86b-fb26b66fca4d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:26.152968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0521da0-f8b8-440a-b86b-fb26b66fca4d None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:26.166706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-c0521da0-f8b8-440a-b86b-fb26b66fca4d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 23 19:53:26.166706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:26.166706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:26.166706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:26.166706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:26.166706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:26.166706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:26.166706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:26.166706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:26.166706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:26.166706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:26.166706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:26.166706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 19:53:26.166706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 19:53:26.166706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 19:53:26.166706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 19:53:26.167661 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:26.167661 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:26.167661 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 19:53:26.167661 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:26.167661 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 19:53:26.167661 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:26.167661 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:26.167661 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:26.167661 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:26.167661 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:26.167661 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:26.167661 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:26.167661 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:26.167661 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:26.167661 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:26.167661 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:26.167661 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:26.167661 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:26.168604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:26.168604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:26.168604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 19:53:26.168604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 19:53:26.168604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 19:53:26.168604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 19:53:26.168604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 19:53:26.168604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 19:53:26.168604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 23 19:53:26.168604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:26.168604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 279/557] 10.222.0.13 () {56 vars in 1117 bytes} [Sun Oct 23 19:53:25 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:26.169185 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0521da0-f8b8-440a-b86b-fb26b66fca4d None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:26.227533 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e07d813-607e-46a5-93d0-d66e7549eaaa None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:26.264098 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe9c42d-efe4-4a85-9080-5cb2ea9efe5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:26.264622 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9c42d-efe4-4a85-9080-5cb2ea9efe5e None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:26.264954 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9c42d-efe4-4a85-9080-5cb2ea9efe5e None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:26.265233 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe9c42d-efe4-4a85-9080-5cb2ea9efe5e None admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:26.265914 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe9c42d-efe4-4a85-9080-5cb2ea9efe5e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:26.266198 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe9c42d-efe4-4a85-9080-5cb2ea9efe5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:26.267827 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe9c42d-efe4-4a85-9080-5cb2ea9efe5e None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:26.281827 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 279/558] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:53:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:26.282881 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe9c42d-efe4-4a85-9080-5cb2ea9efe5e None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:26.309617 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0521da0-f8b8-440a-b86b-fb26b66fca4d None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:26.345400 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:26.356840 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b14a8e-b490-4dc3-ae23-18feaab6c4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:26.358154 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b14a8e-b490-4dc3-ae23-18feaab6c4a0 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:26.359416 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b14a8e-b490-4dc3-ae23-18feaab6c4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:26.359846 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b14a8e-b490-4dc3-ae23-18feaab6c4a0 None admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:26.360880 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b14a8e-b490-4dc3-ae23-18feaab6c4a0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:26.361340 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b14a8e-b490-4dc3-ae23-18feaab6c4a0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:26.363703 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b14a8e-b490-4dc3-ae23-18feaab6c4a0 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:26.379248 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 280/559] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Oct 23 19:53:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:26.380277 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b14a8e-b490-4dc3-ae23-18feaab6c4a0 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:26.393722 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:26.401441 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe9c42d-efe4-4a85-9080-5cb2ea9efe5e None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:26.446616 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f83053-3fc6-4aae-b679-25eb659584a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:26.447234 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f83053-3fc6-4aae-b679-25eb659584a1 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:26.447520 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f83053-3fc6-4aae-b679-25eb659584a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:26.447798 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f83053-3fc6-4aae-b679-25eb659584a1 None admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:26.448540 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f83053-3fc6-4aae-b679-25eb659584a1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:26.448849 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f83053-3fc6-4aae-b679-25eb659584a1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:26.450774 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f83053-3fc6-4aae-b679-25eb659584a1 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:26.474817 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 280/560] 10.222.0.13 () {56 vars in 1149 bytes} [Sun Oct 23 19:53:26 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:26.475682 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f83053-3fc6-4aae-b679-25eb659584a1 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:26.505859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b14a8e-b490-4dc3-ae23-18feaab6c4a0 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:26.549862 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a880b254-066f-4b30-a868-47e7a6d220ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:26.550894 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880b254-066f-4b30-a868-47e7a6d220ae None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:26.551708 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880b254-066f-4b30-a868-47e7a6d220ae None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:26.552376 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880b254-066f-4b30-a868-47e7a6d220ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:26.565341 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a880b254-066f-4b30-a868-47e7a6d220ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:26.566972 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a880b254-066f-4b30-a868-47e7a6d220ae None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:26.591842 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-a880b254-066f-4b30-a868-47e7a6d220ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:26.591842 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:26.591842 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:26.591842 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:26.591842 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:26.591842 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:26.591842 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:26.591842 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:26.591842 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:26.591842 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:26.591842 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:26.591842 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:26.591842 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:26.591842 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:26.591842 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:26.591842 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:26.592865 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:26.592865 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:26.592865 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:26.592865 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:26.592865 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:26.592865 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:26.592865 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:26.592865 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:26.592865 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:26.592865 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:26.592865 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:26.592865 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:26.592865 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:26.592865 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:26.592865 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:26.592865 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:26.592865 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:26.593796 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:26.593796 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:26.593796 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:26.595025 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 281/561] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:26.595025 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a880b254-066f-4b30-a868-47e7a6d220ae None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:26.597268 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f83053-3fc6-4aae-b679-25eb659584a1 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:26.616203 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:26.645231 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef88e4c-b331-43d9-a081-c14d73bed613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:26.646003 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef88e4c-b331-43d9-a081-c14d73bed613 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:26.646791 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef88e4c-b331-43d9-a081-c14d73bed613 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:26.647180 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef88e4c-b331-43d9-a081-c14d73bed613 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:26.648032 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef88e4c-b331-43d9-a081-c14d73bed613 None admin] RBAC: Adding query filter params (role.id=742dfaf63ef74207a2c9213411513925, scope.project.id=d40947427f5f43e7b6c82e3605737767, user.id=0b3d512304254ffcaa8dad90abf1f9c1) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:26.648758 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef88e4c-b331-43d9-a081-c14d73bed613 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:26.650605 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef88e4c-b331-43d9-a081-c14d73bed613 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:26.658278 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 281/562] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Oct 23 19:53:26 2022] GET /identity/v3/role_assignments?user.id=0b3d512304254ffcaa8dad90abf1f9c1&scope.project.id=d40947427f5f43e7b6c82e3605737767&role.id=742dfaf63ef74207a2c9213411513925 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:26.659616 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef88e4c-b331-43d9-a081-c14d73bed613 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:26.667425 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:26.775007 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a880b254-066f-4b30-a868-47e7a6d220ae None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:26.836215 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4013e530-e07f-4157-ad7a-ff3a82b33db9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:26.836913 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013e530-e07f-4157-ad7a-ff3a82b33db9 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:26.837199 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013e530-e07f-4157-ad7a-ff3a82b33db9 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:26.837470 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013e530-e07f-4157-ad7a-ff3a82b33db9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:26.851893 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4013e530-e07f-4157-ad7a-ff3a82b33db9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:26.853503 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4013e530-e07f-4157-ad7a-ff3a82b33db9 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:26.869102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-4013e530-e07f-4157-ad7a-ff3a82b33db9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:26.869102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:26.869102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:26.869102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:26.869102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:26.869102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:26.869102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:26.869102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:26.869102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:26.869102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:26.869102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:26.869102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:26.869102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:26.869102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:26.869102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:26.869102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:26.870345 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:26.870345 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:26.870345 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:26.870345 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:26.870345 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:26.870345 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:26.870345 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:26.870345 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:26.870345 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:26.870345 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:26.870345 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:26.870345 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:26.870345 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:26.870345 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:26.870345 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:26.870345 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:26.870345 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:26.871411 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:26.871411 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:26.871411 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:26.871411 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 282/563] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:26.872878 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef88e4c-b331-43d9-a081-c14d73bed613 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:26.876456 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4013e530-e07f-4157-ad7a-ff3a82b33db9 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:26.914797 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e8876-8f47-4a2d-95f3-4f0b132075e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:26.915986 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e8876-8f47-4a2d-95f3-4f0b132075e7 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:26.916967 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e8876-8f47-4a2d-95f3-4f0b132075e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:26.918520 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e8876-8f47-4a2d-95f3-4f0b132075e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:26.919391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e8876-8f47-4a2d-95f3-4f0b132075e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:26.919901 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e8876-8f47-4a2d-95f3-4f0b132075e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:26.921530 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e8876-8f47-4a2d-95f3-4f0b132075e7 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:26.940117 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 282/564] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:26.947005 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e8876-8f47-4a2d-95f3-4f0b132075e7 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:26.997341 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4013e530-e07f-4157-ad7a-ff3a82b33db9 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:27.003083 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:27.017737 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:27.045026 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef700b21-da40-4edf-a8e8-ec5e62901bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:27.046001 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef700b21-da40-4edf-a8e8-ec5e62901bb0 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:27.047447 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef700b21-da40-4edf-a8e8-ec5e62901bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:27.047922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef700b21-da40-4edf-a8e8-ec5e62901bb0 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:27.049916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef700b21-da40-4edf-a8e8-ec5e62901bb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:27.050469 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef700b21-da40-4edf-a8e8-ec5e62901bb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:27.052072 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef700b21-da40-4edf-a8e8-ec5e62901bb0 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:27.067849 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 283/565] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:53:27.074543 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef700b21-da40-4edf-a8e8-ec5e62901bb0 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:27.097209 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e8876-8f47-4a2d-95f3-4f0b132075e7 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:27.164058 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465019b9-4156-4bd8-b0cf-d5c0a0260f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:27.165080 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465019b9-4156-4bd8-b0cf-d5c0a0260f30 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:27.165562 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465019b9-4156-4bd8-b0cf-d5c0a0260f30 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:27.166033 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465019b9-4156-4bd8-b0cf-d5c0a0260f30 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:27.176646 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465019b9-4156-4bd8-b0cf-d5c0a0260f30 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:27.179846 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465019b9-4156-4bd8-b0cf-d5c0a0260f30 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:27.205197 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-465019b9-4156-4bd8-b0cf-d5c0a0260f30 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 23 19:53:27.205197 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:27.205197 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:27.205197 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:27.205197 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:27.205197 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:27.205197 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:27.205197 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:27.205197 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:27.205197 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:27.205197 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:27.205197 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:27.205197 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 19:53:27.205197 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 19:53:27.205197 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 19:53:27.205197 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 19:53:27.206175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:27.206175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:27.206175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 19:53:27.206175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:27.206175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 19:53:27.206175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:27.206175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:27.206175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:27.206175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:27.206175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:27.206175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:27.206175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:27.206175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:27.206175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:27.206175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:27.206175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:27.206175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:27.206175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:27.207157 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:27.207157 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:27.207157 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 19:53:27.207157 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 19:53:27.207157 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 19:53:27.207157 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 19:53:27.207157 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 19:53:27.207157 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 19:53:27.207157 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 23 19:53:27.207157 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:27.213381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 283/566] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Oct 23 19:53:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:53:27.218641 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465019b9-4156-4bd8-b0cf-d5c0a0260f30 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:27.280677 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef700b21-da40-4edf-a8e8-ec5e62901bb0 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:27.340190 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f788c-0c28-4e69-b9ad-ba1a68ff294f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:27.342036 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f788c-0c28-4e69-b9ad-ba1a68ff294f None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:27.343856 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f788c-0c28-4e69-b9ad-ba1a68ff294f None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:27.344368 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f788c-0c28-4e69-b9ad-ba1a68ff294f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:27.361794 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f788c-0c28-4e69-b9ad-ba1a68ff294f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:27.363737 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f788c-0c28-4e69-b9ad-ba1a68ff294f None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:27.388365 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-744f788c-0c28-4e69-b9ad-ba1a68ff294f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:27.388365 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:27.388365 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:27.388365 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:27.388365 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:27.388365 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:27.388365 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:27.388365 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:27.388365 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:27.388365 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:27.388365 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:27.388365 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:27.388365 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 19:53:27.388365 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 19:53:27.388365 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 19:53:27.389419 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 19:53:27.389419 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:27.389419 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:27.389419 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:27.389419 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:27.389419 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:27.389419 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:27.389419 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:27.389419 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:27.389419 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:27.389419 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:27.389419 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:27.389419 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:27.389419 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:27.389419 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:27.389419 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:27.389419 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:27.390911 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 19:53:27.390911 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 19:53:27.390911 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 19:53:27.390911 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 19:53:27.390911 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 19:53:27.390911 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 19:53:27.390911 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:27.390911 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:27.394447 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 284/567] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:53:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:27.406706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f788c-0c28-4e69-b9ad-ba1a68ff294f None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:27.420770 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:27.433160 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465019b9-4156-4bd8-b0cf-d5c0a0260f30 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:27.452101 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:27.510946 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aaefd2-9bbc-48e2-bab4-4c6695768dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:27.511941 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aaefd2-9bbc-48e2-bab4-4c6695768dde None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:27.515457 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aaefd2-9bbc-48e2-bab4-4c6695768dde None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:27.515970 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aaefd2-9bbc-48e2-bab4-4c6695768dde None admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:27.516933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aaefd2-9bbc-48e2-bab4-4c6695768dde None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:27.517443 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aaefd2-9bbc-48e2-bab4-4c6695768dde None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:27.519119 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aaefd2-9bbc-48e2-bab4-4c6695768dde None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:27.536292 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f788c-0c28-4e69-b9ad-ba1a68ff294f None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:27.540906 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 284/568] 10.222.0.13 () {56 vars in 1135 bytes} [Sun Oct 23 19:53:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:27.546860 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec94829-56e3-4b60-8078-a5f51c7e219c None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:27.547409 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec94829-56e3-4b60-8078-a5f51c7e219c None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:27.547891 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec94829-56e3-4b60-8078-a5f51c7e219c None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:27.549667 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 285/569] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:27 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:27.550616 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec94829-56e3-4b60-8078-a5f51c7e219c None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:27.581779 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c86d61e-770a-4762-9c1e-35e621e90a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:27.582457 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86d61e-770a-4762-9c1e-35e621e90a21 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:27.582854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86d61e-770a-4762-9c1e-35e621e90a21 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:27.583110 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c86d61e-770a-4762-9c1e-35e621e90a21 None admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:27.583816 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86d61e-770a-4762-9c1e-35e621e90a21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:27.584385 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86d61e-770a-4762-9c1e-35e621e90a21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:27.585688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c86d61e-770a-4762-9c1e-35e621e90a21 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:27.600764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 285/570] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Oct 23 19:53:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:53:27.602434 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51b35a-871b-40c8-bb1c-a96162436657 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:27.602922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51b35a-871b-40c8-bb1c-a96162436657 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:27.603391 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c51b35a-871b-40c8-bb1c-a96162436657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:27.623896 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-2c51b35a-871b-40c8-bb1c-a96162436657 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:27.625915 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2c51b35a-871b-40c8-bb1c-a96162436657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:27.678462 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec94829-56e3-4b60-8078-a5f51c7e219c None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:27.708447 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 286/571] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:27 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:53:27.709754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51b35a-871b-40c8-bb1c-a96162436657 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:27.740421 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e6020-ab2d-4aca-9671-c4c65e43a966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:27.741408 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e6020-ab2d-4aca-9671-c4c65e43a966 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:27.741913 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e6020-ab2d-4aca-9671-c4c65e43a966 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:27.742427 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e6020-ab2d-4aca-9671-c4c65e43a966 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:27.758479 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:27.760333 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e6020-ab2d-4aca-9671-c4c65e43a966 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:27.761779 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e6020-ab2d-4aca-9671-c4c65e43a966 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:27.792268 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-fb0e6020-ab2d-4aca-9671-c4c65e43a966 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:27.792268 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:27.792268 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:27.792268 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:27.792268 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:27.792268 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:27.792268 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:27.792268 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:27.792268 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:27.792268 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:27.792268 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:27.792268 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:27.792268 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:27.792268 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:27.792268 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:27.792268 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:27.793359 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:27.793359 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:27.793359 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:27.793359 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:27.793359 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:27.793359 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:27.793359 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:27.793359 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:27.793359 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:27.793359 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:27.793359 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:27.793359 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:27.793359 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:27.793359 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:27.793359 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:27.793359 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:27.793359 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:27.794308 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:27.794308 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:27.794308 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:27.794308 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 286/572] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:27.799618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e6020-ab2d-4aca-9671-c4c65e43a966 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:27.816287 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:27.914733 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e6020-ab2d-4aca-9671-c4c65e43a966 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:27.920878 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c51b35a-871b-40c8-bb1c-a96162436657 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:27.957039 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f65ffd-39fb-4e0b-8c5d-be34e4d83f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:27.957672 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f65ffd-39fb-4e0b-8c5d-be34e4d83f59 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:27.957953 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f65ffd-39fb-4e0b-8c5d-be34e4d83f59 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:27.958225 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f65ffd-39fb-4e0b-8c5d-be34e4d83f59 None admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:27.958922 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f65ffd-39fb-4e0b-8c5d-be34e4d83f59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:27.959227 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f65ffd-39fb-4e0b-8c5d-be34e4d83f59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:27.960661 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f65ffd-39fb-4e0b-8c5d-be34e4d83f59 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:27.988308 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 287/573] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:27.994558 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f65ffd-39fb-4e0b-8c5d-be34e4d83f59 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:27.999254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315d64d-a7ad-4217-a062-e8095b2fc12b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:27.999927 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315d64d-a7ad-4217-a062-e8095b2fc12b None admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:28.000208 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315d64d-a7ad-4217-a062-e8095b2fc12b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:28.000481 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1315d64d-a7ad-4217-a062-e8095b2fc12b None admin] PATH_INFO: `/v3/projects/d40947427f5f43e7b6c82e3605737767/users/ff5cfe4d31614e2db9059c69116297cc/roles/5e0524ede3ee479c923c7b609a0ca402` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:28.028585 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1315d64d-a7ad-4217-a062-e8095b2fc12b None admin] RBAC: Authorizing `identity:create_grant(project_id=d40947427f5f43e7b6c82e3605737767, user_id=ff5cfe4d31614e2db9059c69116297cc, role_id=5e0524ede3ee479c923c7b609a0ca402)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:28.030051 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1315d64d-a7ad-4217-a062-e8095b2fc12b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:28.058635 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 287/574] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Oct 23 19:53:27 2022] PUT /identity/v3/projects/d40947427f5f43e7b6c82e3605737767/users/ff5cfe4d31614e2db9059c69116297cc/roles/5e0524ede3ee479c923c7b609a0ca402 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 19:53:28.060657 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315d64d-a7ad-4217-a062-e8095b2fc12b None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:28.114839 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:28.166401 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:28.173828 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f65ffd-39fb-4e0b-8c5d-be34e4d83f59 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:28.202057 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1315d64d-a7ad-4217-a062-e8095b2fc12b None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:28.228319 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26addb79-2ea9-49d9-b4b2-ae8419535d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:28.229937 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26addb79-2ea9-49d9-b4b2-ae8419535d63 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:28.230439 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26addb79-2ea9-49d9-b4b2-ae8419535d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:28.231159 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26addb79-2ea9-49d9-b4b2-ae8419535d63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:28.240736 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26addb79-2ea9-49d9-b4b2-ae8419535d63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:28.242076 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca028b-539f-4751-8958-cbbb779a35cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:28.242226 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26addb79-2ea9-49d9-b4b2-ae8419535d63 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:28.242973 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca028b-539f-4751-8958-cbbb779a35cb None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:28.243314 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca028b-539f-4751-8958-cbbb779a35cb None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:28.243592 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ca028b-539f-4751-8958-cbbb779a35cb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:28.244178 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ca028b-539f-4751-8958-cbbb779a35cb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:28.245512 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ca028b-539f-4751-8958-cbbb779a35cb None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:28.254489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-26addb79-2ea9-49d9-b4b2-ae8419535d63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:28.254489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:28.254489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:28.254489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:28.254489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:28.254489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:28.254489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:28.254489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:28.254489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:28.254489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:28.254489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:28.254489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:28.254489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 19:53:28.254489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 19:53:28.254489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 19:53:28.255406 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 19:53:28.255406 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:28.255406 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:28.255406 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:28.255406 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:28.255406 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:28.255406 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:28.255406 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:28.255406 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:28.255406 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:28.255406 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:28.255406 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:28.255406 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:28.255406 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:28.255406 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:28.255406 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:28.255406 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:28.256290 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 19:53:28.256290 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 19:53:28.256290 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 19:53:28.256290 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 19:53:28.256290 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 19:53:28.256290 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 19:53:28.256290 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:28.256290 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:28.256290 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 288/575] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:53:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:28.256885 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-d5ca028b-539f-4751-8958-cbbb779a35cb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 23 19:53:28.256885 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:28.256885 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 19:53:28.256885 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 19:53:28.256885 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 19:53:28.256885 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 19:53:28.256885 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 19:53:28.256885 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:28.256885 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 23 19:53:28.256885 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:28.256885 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 19:53:28.256885 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:28.256885 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:28.256885 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:28.256885 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:28.256885 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:28.256885 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:28.257782 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:28.257782 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:28.257782 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:28.257782 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:28.257782 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:28.257782 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:28.257782 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 19:53:28.257782 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 19:53:28.257782 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 19:53:28.257782 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 19:53:28.257782 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:28.257782 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:28.257782 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:28.257782 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:28.257782 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:28.257782 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:28.259952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:28.259952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:28.259952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:28.259952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:28.259952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:28.259952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:28.259952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:28.259952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:28.259952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:28.259952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:28.259952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 19:53:28.259952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:28.259952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 23 19:53:28.259952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:28.261803 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3315b7f-e719-4116-92f2-f660a0045978 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:28.261943 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 288/576] 10.222.0.13 () {56 vars in 1129 bytes} [Sun Oct 23 19:53:28 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:28.262078 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3315b7f-e719-4116-92f2-f660a0045978 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:28.262078 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3315b7f-e719-4116-92f2-f660a0045978 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:28.262078 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 289/577] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:28.263818 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca028b-539f-4751-8958-cbbb779a35cb None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:28.263974 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3315b7f-e719-4116-92f2-f660a0045978 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:28.385413 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ca028b-539f-4751-8958-cbbb779a35cb None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:28.391080 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3315b7f-e719-4116-92f2-f660a0045978 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:28.419443 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:28.428100 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50a0a4b-54f6-4e2a-aaf2-2f3add683d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:28.428536 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac88617-c0e8-4981-ae7e-0f1fb768f259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:28.428700 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a0a4b-54f6-4e2a-aaf2-2f3add683d58 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:28.428982 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a0a4b-54f6-4e2a-aaf2-2f3add683d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:28.429251 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50a0a4b-54f6-4e2a-aaf2-2f3add683d58 None admin] PATH_INFO: `/v3/services` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:28.429762 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac88617-c0e8-4981-ae7e-0f1fb768f259 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:28.429875 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a0a4b-54f6-4e2a-aaf2-2f3add683d58 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:28.430170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a0a4b-54f6-4e2a-aaf2-2f3add683d58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:28.430452 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac88617-c0e8-4981-ae7e-0f1fb768f259 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:28.430888 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac88617-c0e8-4981-ae7e-0f1fb768f259 None admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:28.431606 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50a0a4b-54f6-4e2a-aaf2-2f3add683d58 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:28.431881 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac88617-c0e8-4981-ae7e-0f1fb768f259 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:28.432329 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac88617-c0e8-4981-ae7e-0f1fb768f259 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:28.434028 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac88617-c0e8-4981-ae7e-0f1fb768f259 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:28.441912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 290/578] 10.222.0.13 () {56 vars in 1124 bytes} [Sun Oct 23 19:53:28 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:53:28.441912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42fab4-c320-404c-a131-53729342518c None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:28.441912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42fab4-c320-404c-a131-53729342518c None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:28.441912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c42fab4-c320-404c-a131-53729342518c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:28.442797 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:28.456100 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 289/579] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Oct 23 19:53:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:28.456954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac88617-c0e8-4981-ae7e-0f1fb768f259 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:28.462189 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-1c42fab4-c320-404c-a131-53729342518c None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:28.463605 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-1c42fab4-c320-404c-a131-53729342518c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:28.542447 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 291/580] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:53:28.543666 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c42fab4-c320-404c-a131-53729342518c None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:28.570268 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac88617-c0e8-4981-ae7e-0f1fb768f259 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:28.606320 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e267618-c93b-4c33-8d60-38b27e302de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:28.607202 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e267618-c93b-4c33-8d60-38b27e302de4 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:28.607492 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e267618-c93b-4c33-8d60-38b27e302de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:28.607771 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e267618-c93b-4c33-8d60-38b27e302de4 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:28.609343 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e267618-c93b-4c33-8d60-38b27e302de4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:28.609653 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e267618-c93b-4c33-8d60-38b27e302de4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:28.611179 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e267618-c93b-4c33-8d60-38b27e302de4 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:28.617369 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 290/581] 10.222.0.13 () {56 vars in 1124 bytes} [Sun Oct 23 19:53:28 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:28.622805 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e267618-c93b-4c33-8d60-38b27e302de4 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:28.694818 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:28.731540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e267618-c93b-4c33-8d60-38b27e302de4 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:28.770011 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405df21-56ef-4f78-961a-3ded4e0ef395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:28.770680 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405df21-56ef-4f78-961a-3ded4e0ef395 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:28.770965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405df21-56ef-4f78-961a-3ded4e0ef395 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:28.771248 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405df21-56ef-4f78-961a-3ded4e0ef395 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:28.771961 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b405df21-56ef-4f78-961a-3ded4e0ef395 None admin] RBAC: Adding query filter params (interface=public, service_id=df045cc47fca41958690908f6b20c8ce, region_id=RegionOne) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:28.772278 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b405df21-56ef-4f78-961a-3ded4e0ef395 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:28.773682 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b405df21-56ef-4f78-961a-3ded4e0ef395 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:28.782146 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 291/582] 10.222.0.13 () {56 vars in 1251 bytes} [Sun Oct 23 19:53:28 2022] GET /identity/v3/endpoints?service_id=df045cc47fca41958690908f6b20c8ce&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:28.791821 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405df21-56ef-4f78-961a-3ded4e0ef395 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:28.831767 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c42fab4-c320-404c-a131-53729342518c None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:28.874880 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:28.894951 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d93040-2c3d-4b57-bf13-4b9c2446d2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:28.895624 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d93040-2c3d-4b57-bf13-4b9c2446d2e2 None admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:28.895904 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d93040-2c3d-4b57-bf13-4b9c2446d2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:28.896179 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d93040-2c3d-4b57-bf13-4b9c2446d2e2 None admin] PATH_INFO: `/v3/projects/d40947427f5f43e7b6c82e3605737767/users/15f603d1e7f54751aa79317e2ea00a31/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:28.901696 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d93040-2c3d-4b57-bf13-4b9c2446d2e2 None admin] RBAC: Authorizing `identity:create_grant(project_id=d40947427f5f43e7b6c82e3605737767, user_id=15f603d1e7f54751aa79317e2ea00a31, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:28.903168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d93040-2c3d-4b57-bf13-4b9c2446d2e2 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:28.914615 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405df21-56ef-4f78-961a-3ded4e0ef395 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:28.920734 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 292/583] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Oct 23 19:53:28 2022] PUT /identity/v3/projects/d40947427f5f43e7b6c82e3605737767/users/15f603d1e7f54751aa79317e2ea00a31/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 19:53:28.926322 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d93040-2c3d-4b57-bf13-4b9c2446d2e2 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:28.966343 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:28.995541 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de528d22-9ff6-442c-8e6e-6db8ec02d9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:28.996835 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de528d22-9ff6-442c-8e6e-6db8ec02d9f4 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:28.997364 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de528d22-9ff6-442c-8e6e-6db8ec02d9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:28.997838 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de528d22-9ff6-442c-8e6e-6db8ec02d9f4 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:28.998689 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de528d22-9ff6-442c-8e6e-6db8ec02d9f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:29.000823 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de528d22-9ff6-442c-8e6e-6db8ec02d9f4 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:29.015194 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 292/584] 10.222.0.13 () {56 vars in 1087 bytes} [Sun Oct 23 19:53:28 2022] GET /identity/v3/services => generated 772 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:53:29.088766 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d93040-2c3d-4b57-bf13-4b9c2446d2e2 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:29.129382 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa82d8d-2b04-4019-90a3-7a11a04fa5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:29.130032 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa82d8d-2b04-4019-90a3-7a11a04fa5cd None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:29.130343 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa82d8d-2b04-4019-90a3-7a11a04fa5cd None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:29.130615 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa82d8d-2b04-4019-90a3-7a11a04fa5cd None admin] PATH_INFO: `/v3/services/network` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:29.131222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa82d8d-2b04-4019-90a3-7a11a04fa5cd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:29.132663 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa82d8d-2b04-4019-90a3-7a11a04fa5cd None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:29.144725 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-0fa82d8d-2b04-4019-90a3-7a11a04fa5cd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 23 19:53:29.144725 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:29.144725 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 19:53:29.144725 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 19:53:29.144725 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 19:53:29.144725 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 19:53:29.144725 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 19:53:29.144725 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:29.144725 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 23 19:53:29.144725 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:29.144725 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 19:53:29.144725 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:29.144725 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:29.144725 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:29.144725 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:29.144725 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:29.144725 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:29.145864 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:29.145864 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:29.145864 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:29.145864 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:29.145864 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:29.145864 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:29.145864 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 19:53:29.145864 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 19:53:29.145864 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 19:53:29.145864 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 19:53:29.145864 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:29.145864 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:29.145864 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:29.145864 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:29.145864 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:29.145864 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:29.146787 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:29.146787 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:29.146787 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:29.146787 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:29.146787 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:29.146787 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:29.146787 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:29.146787 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:29.146787 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:29.146787 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:29.146787 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 19:53:29.146787 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:29.146787 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 23 19:53:29.146787 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:29.150369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 293/585] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:53:28 2022] GET /identity/v3/services/network => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:29.152331 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de528d22-9ff6-442c-8e6e-6db8ec02d9f4 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:29.272701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de528d22-9ff6-442c-8e6e-6db8ec02d9f4 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:29.308491 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f944d80d-0927-4fc6-a67e-8be4de7e0281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:29.309477 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f944d80d-0927-4fc6-a67e-8be4de7e0281 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:29.309968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f944d80d-0927-4fc6-a67e-8be4de7e0281 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:29.310479 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f944d80d-0927-4fc6-a67e-8be4de7e0281 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:29.311362 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f944d80d-0927-4fc6-a67e-8be4de7e0281 None admin] RBAC: Adding query filter params (name=network) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:29.311976 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f944d80d-0927-4fc6-a67e-8be4de7e0281 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:29.313548 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f944d80d-0927-4fc6-a67e-8be4de7e0281 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:29.318942 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 293/586] 10.222.0.13 () {56 vars in 1112 bytes} [Sun Oct 23 19:53:29 2022] GET /identity/v3/services?name=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:29.324314 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa82d8d-2b04-4019-90a3-7a11a04fa5cd None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:29.378575 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:29.440899 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa82d8d-2b04-4019-90a3-7a11a04fa5cd None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:29.481351 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945584e8-4127-4c20-9ed7-d0a6553d5f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:29.481967 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945584e8-4127-4c20-9ed7-d0a6553d5f62 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:29.482261 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945584e8-4127-4c20-9ed7-d0a6553d5f62 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:29.482545 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945584e8-4127-4c20-9ed7-d0a6553d5f62 None admin] PATH_INFO: `/v3/services` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:29.483190 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945584e8-4127-4c20-9ed7-d0a6553d5f62 None admin] RBAC: Adding query filter params (type=network) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:29.483492 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945584e8-4127-4c20-9ed7-d0a6553d5f62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:29.485406 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945584e8-4127-4c20-9ed7-d0a6553d5f62 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:29.494639 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 294/587] 10.222.0.13 () {56 vars in 1112 bytes} [Sun Oct 23 19:53:29 2022] GET /identity/v3/services?type=network => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:29.500582 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f944d80d-0927-4fc6-a67e-8be4de7e0281 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:29.512506 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5acddc6-7239-46c5-8114-91e4214fbeba None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:29.513090 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5acddc6-7239-46c5-8114-91e4214fbeba None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:29.514076 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5acddc6-7239-46c5-8114-91e4214fbeba None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:29.516494 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 295/588] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:29.522089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a194c7-9d8e-43e9-9a9e-9155e83ad04f None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:29.522655 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a194c7-9d8e-43e9-9a9e-9155e83ad04f None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:29.523355 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a194c7-9d8e-43e9-9a9e-9155e83ad04f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:29.542887 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-a2a194c7-9d8e-43e9-9a9e-9155e83ad04f None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:29.544830 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-a2a194c7-9d8e-43e9-9a9e-9155e83ad04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:29.620443 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 296/589] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:53:29.626635 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:29.679820 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f944d80d-0927-4fc6-a67e-8be4de7e0281 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:29.714017 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf231d94-b074-4e69-be0e-4ad4f7722704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:29.714692 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf231d94-b074-4e69-be0e-4ad4f7722704 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:29.714970 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf231d94-b074-4e69-be0e-4ad4f7722704 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:29.715238 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf231d94-b074-4e69-be0e-4ad4f7722704 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:29.715936 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf231d94-b074-4e69-be0e-4ad4f7722704 None admin] RBAC: Adding query filter params (interface=public, service_id=50204a592a7849ed9acf450884ad8f3a, region_id=RegionOne) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:29.716238 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf231d94-b074-4e69-be0e-4ad4f7722704 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:29.717580 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf231d94-b074-4e69-be0e-4ad4f7722704 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:29.728428 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 294/590] 10.222.0.13 () {56 vars in 1251 bytes} [Sun Oct 23 19:53:29 2022] GET /identity/v3/endpoints?service_id=50204a592a7849ed9acf450884ad8f3a&interface=public®ion_id=RegionOne => generated 194 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:29.730147 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a194c7-9d8e-43e9-9a9e-9155e83ad04f None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:29.790048 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:29.867460 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a194c7-9d8e-43e9-9a9e-9155e83ad04f None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:29.911495 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289282a7-5f10-4f94-9c58-164a530e3524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:29.912162 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289282a7-5f10-4f94-9c58-164a530e3524 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:29.912438 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289282a7-5f10-4f94-9c58-164a530e3524 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:29.912754 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289282a7-5f10-4f94-9c58-164a530e3524 None admin] PATH_INFO: `/v3/services` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:29.913370 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289282a7-5f10-4f94-9c58-164a530e3524 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:29.914862 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289282a7-5f10-4f94-9c58-164a530e3524 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:29.923993 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 297/591] 10.222.0.13 () {56 vars in 1087 bytes} [Sun Oct 23 19:53:29 2022] GET /identity/v3/services => generated 772 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:53:29.982642 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf231d94-b074-4e69-be0e-4ad4f7722704 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:30.102666 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf231d94-b074-4e69-be0e-4ad4f7722704 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:30.128219 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:30.144105 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cfe2d2-4d5a-4531-a665-17991387080e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:30.144924 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cfe2d2-4d5a-4531-a665-17991387080e None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:30.145299 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cfe2d2-4d5a-4531-a665-17991387080e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:30.145667 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cfe2d2-4d5a-4531-a665-17991387080e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:30.146402 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cfe2d2-4d5a-4531-a665-17991387080e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:30.147872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cfe2d2-4d5a-4531-a665-17991387080e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:30.157822 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-56cfe2d2-4d5a-4531-a665-17991387080e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 23 19:53:30.157822 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:30.157822 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 19:53:30.157822 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 19:53:30.157822 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 19:53:30.157822 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 19:53:30.157822 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 19:53:30.157822 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:30.157822 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 23 19:53:30.157822 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:30.157822 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 19:53:30.157822 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:30.157822 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:30.157822 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:30.157822 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:30.157822 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:30.158903 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:30.158903 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:30.158903 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:30.158903 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:30.158903 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:30.158903 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:30.158903 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:30.158903 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 19:53:30.158903 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 19:53:30.158903 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 19:53:30.158903 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 19:53:30.158903 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:30.158903 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:30.158903 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:30.158903 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:30.158903 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:30.158903 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:30.159789 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:30.159789 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:30.159789 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:30.159789 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:30.159789 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:30.159789 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:30.159789 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:30.159789 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:30.159789 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:30.159789 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:30.159789 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 19:53:30.159789 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:30.159789 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 23 19:53:30.159789 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:30.159789 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 295/592] 10.222.0.13 () {56 vars in 1132 bytes} [Sun Oct 23 19:53:29 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:30.166633 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289282a7-5f10-4f94-9c58-164a530e3524 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:30.303145 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289282a7-5f10-4f94-9c58-164a530e3524 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:30.306485 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:30.347959 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583e8839-e430-4b20-a8fc-e51ff7ed10e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:30.348981 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e8839-e430-4b20-a8fc-e51ff7ed10e0 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:30.349468 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e8839-e430-4b20-a8fc-e51ff7ed10e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:30.350107 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583e8839-e430-4b20-a8fc-e51ff7ed10e0 None admin] PATH_INFO: `/v3/services` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:30.350971 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e8839-e430-4b20-a8fc-e51ff7ed10e0 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:30.351477 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e8839-e430-4b20-a8fc-e51ff7ed10e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:30.353164 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583e8839-e430-4b20-a8fc-e51ff7ed10e0 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:30.358138 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 298/593] 10.222.0.13 () {56 vars in 1126 bytes} [Sun Oct 23 19:53:30 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:30.364674 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cfe2d2-4d5a-4531-a665-17991387080e None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:30.504101 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cfe2d2-4d5a-4531-a665-17991387080e None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:30.574965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f4f2a-51c5-4650-b8e0-eef6b2bdeb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:30.575595 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f4f2a-51c5-4650-b8e0-eef6b2bdeb15 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:30.575874 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f4f2a-51c5-4650-b8e0-eef6b2bdeb15 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:30.576145 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f4f2a-51c5-4650-b8e0-eef6b2bdeb15 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:30.576785 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f4f2a-51c5-4650-b8e0-eef6b2bdeb15 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:30.577084 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f4f2a-51c5-4650-b8e0-eef6b2bdeb15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:30.578548 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f4f2a-51c5-4650-b8e0-eef6b2bdeb15 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:30.583867 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 296/594] 10.222.0.13 () {56 vars in 1126 bytes} [Sun Oct 23 19:53:30 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:30.659963 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b768d-74ce-4e1b-8751-c7100aa63435 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:30.660574 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b768d-74ce-4e1b-8751-c7100aa63435 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:30.661057 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b768d-74ce-4e1b-8751-c7100aa63435 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:30.662682 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 299/595] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:30.675127 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819afd1-e58d-42b4-9292-952726a39755 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:30.675449 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819afd1-e58d-42b4-9292-952726a39755 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:30.675670 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819afd1-e58d-42b4-9292-952726a39755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:30.699325 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-8819afd1-e58d-42b4-9292-952726a39755 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:30.700547 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-8819afd1-e58d-42b4-9292-952726a39755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:30.774658 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:30.823675 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 297/596] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:53:31.261712 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b768d-74ce-4e1b-8751-c7100aa63435 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:31.403573 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b768d-74ce-4e1b-8751-c7100aa63435 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:31.451892 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a956605c-04b7-4ee3-aa16-06aeb1e80572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:31.452980 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956605c-04b7-4ee3-aa16-06aeb1e80572 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:31.453479 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956605c-04b7-4ee3-aa16-06aeb1e80572 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:31.453954 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956605c-04b7-4ee3-aa16-06aeb1e80572 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:31.469675 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a956605c-04b7-4ee3-aa16-06aeb1e80572 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:53:31.474996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-a956605c-04b7-4ee3-aa16-06aeb1e80572 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 19:53:31.474996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:31.474996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:31.474996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:31.474996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:31.474996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:31.474996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:31.474996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:31.474996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:31.474996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:31.474996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:31.474996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:31.474996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:53:31.474996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:53:31.474996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:53:31.474996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:53:31.475995 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:53:31.475995 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:53:31.475995 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:53:31.475995 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:53:31.475995 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:31.475995 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:31.475995 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:31.475995 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:31.475995 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:31.475995 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:31.475995 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:31.475995 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:31.475995 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:31.475995 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:31.475995 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:31.475995 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:31.476903 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:31.476903 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:31.476903 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:31.476903 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:31.476903 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:53:31.476903 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:53:31.476903 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:53:31.476903 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:53:31.476903 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:53:31.476903 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:53:31.476903 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 19:53:31.476903 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:31.479301 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 300/597] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Oct 23 19:53:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:31.485469 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819afd1-e58d-42b4-9292-952726a39755 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:31.636322 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819afd1-e58d-42b4-9292-952726a39755 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:31.678284 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a1ccc8-1993-460f-9832-2480741823ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:31.678859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1ccc8-1993-460f-9832-2480741823ef None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:31.679137 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1ccc8-1993-460f-9832-2480741823ef None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:31.679406 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a1ccc8-1993-460f-9832-2480741823ef None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:31.680066 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a1ccc8-1993-460f-9832-2480741823ef None admin] RBAC: Adding query filter params (name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:31.680372 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a1ccc8-1993-460f-9832-2480741823ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:31.681736 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a1ccc8-1993-460f-9832-2480741823ef None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:31.690430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 298/598] 10.222.0.13 () {56 vars in 1103 bytes} [Sun Oct 23 19:53:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:31.695845 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a956605c-04b7-4ee3-aa16-06aeb1e80572 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:31.741404 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aed2fb-5195-4fe5-910c-4671f5126168 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:31.741404 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aed2fb-5195-4fe5-910c-4671f5126168 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:31.741404 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aed2fb-5195-4fe5-910c-4671f5126168 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:31.741404 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 299/599] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 19:53:31.742906 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:31.749392 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f9754-55f1-49d7-8c7a-c5ea7dee80fa None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:31.749682 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f9754-55f1-49d7-8c7a-c5ea7dee80fa None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:31.749958 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f9754-55f1-49d7-8c7a-c5ea7dee80fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:31.783313 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-3d6f9754-55f1-49d7-8c7a-c5ea7dee80fa None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:31.786422 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3d6f9754-55f1-49d7-8c7a-c5ea7dee80fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:31.846642 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:31.865534 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a956605c-04b7-4ee3-aa16-06aeb1e80572 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:31.916015 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 300/600] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 19:53:31.935042 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573635e9-6948-4d84-86e7-2799b20923bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:31.935625 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573635e9-6948-4d84-86e7-2799b20923bf None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:31.935907 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573635e9-6948-4d84-86e7-2799b20923bf None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:31.936180 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573635e9-6948-4d84-86e7-2799b20923bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:31.948490 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573635e9-6948-4d84-86e7-2799b20923bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:31.949935 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573635e9-6948-4d84-86e7-2799b20923bf None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:31.960307 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e971b7e-d654-480b-87c1-962a5730e7c7 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:31.960664 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e971b7e-d654-480b-87c1-962a5730e7c7 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:31.961019 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e971b7e-d654-480b-87c1-962a5730e7c7 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:31.966266 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 301/601] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 19:53:31.968448 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f889a6-9567-49ba-aa95-d626e108e3c2 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:31.968737 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f889a6-9567-49ba-aa95-d626e108e3c2 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:31.969020 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f889a6-9567-49ba-aa95-d626e108e3c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:31.976039 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-573635e9-6948-4d84-86e7-2799b20923bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:31.976039 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:31.976039 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:31.976039 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:31.976039 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:31.976039 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:31.976039 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:31.976039 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:31.976039 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:31.976039 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:31.976039 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:31.976039 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:31.976039 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:31.976039 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:31.976039 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:31.976039 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:31.976997 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:31.976997 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:31.976997 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:31.976997 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:31.976997 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:31.976997 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:31.976997 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:31.976997 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:31.976997 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:31.976997 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:31.976997 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:31.976997 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:31.976997 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:31.976997 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:31.976997 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:31.976997 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:31.976997 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:31.977896 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:31.977896 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:31.977896 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:31.980024 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 301/602] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:31.991417 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573635e9-6948-4d84-86e7-2799b20923bf None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:31.997131 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-16f889a6-9567-49ba-aa95-d626e108e3c2 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:32.003678 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-16f889a6-9567-49ba-aa95-d626e108e3c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:32.146029 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 302/603] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:53:32.170625 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:32.182652 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573635e9-6948-4d84-86e7-2799b20923bf None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:32.237840 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918bb82-e752-4533-8f80-ae6229f9ab8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:32.238407 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918bb82-e752-4533-8f80-ae6229f9ab8f None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:32.238749 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918bb82-e752-4533-8f80-ae6229f9ab8f None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:32.239032 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918bb82-e752-4533-8f80-ae6229f9ab8f None admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:32.239697 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6918bb82-e752-4533-8f80-ae6229f9ab8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:32.240004 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6918bb82-e752-4533-8f80-ae6229f9ab8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:32.241377 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6918bb82-e752-4533-8f80-ae6229f9ab8f None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:32.248590 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f889a6-9567-49ba-aa95-d626e108e3c2 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:32.271902 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 302/604] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:32.276236 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918bb82-e752-4533-8f80-ae6229f9ab8f None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:32.442610 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:32.450207 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f889a6-9567-49ba-aa95-d626e108e3c2 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:32.481296 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918bb82-e752-4533-8f80-ae6229f9ab8f None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:32.516717 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e904e5a-4698-4224-8266-ca42b455a3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:32.517821 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e904e5a-4698-4224-8266-ca42b455a3f0 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:32.518487 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e904e5a-4698-4224-8266-ca42b455a3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:32.518983 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e904e5a-4698-4224-8266-ca42b455a3f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:32.530402 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb732f5b-0048-42dc-a191-937f3412789b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:32.531717 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb732f5b-0048-42dc-a191-937f3412789b None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:32.531997 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb732f5b-0048-42dc-a191-937f3412789b None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:32.532275 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb732f5b-0048-42dc-a191-937f3412789b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:32.532403 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e904e5a-4698-4224-8266-ca42b455a3f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:53:32.537747 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-0e904e5a-4698-4224-8266-ca42b455a3f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:53:32.537747 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:32.537747 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:32.537747 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:32.537747 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:32.537747 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:32.537747 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:32.537747 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:32.537747 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:32.537747 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:32.537747 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:32.537747 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:32.537747 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:53:32.537747 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:53:32.537747 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:53:32.537747 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:53:32.538695 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:53:32.538695 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:53:32.538695 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:53:32.538695 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:53:32.538695 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:32.538695 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:32.538695 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:32.538695 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:32.538695 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:32.538695 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:32.538695 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:32.538695 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:32.538695 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:32.538695 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:32.538695 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:32.538695 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:32.539558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:32.539558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:32.539558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:32.539558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:32.539558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:53:32.539558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:53:32.539558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:53:32.539558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:53:32.539558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:53:32.539558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:53:32.539558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:53:32.539558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:32.540193 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb732f5b-0048-42dc-a191-937f3412789b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:32.541162 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb732f5b-0048-42dc-a191-937f3412789b None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:32.547151 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 303/605] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Oct 23 19:53:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:53:32.547913 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:32.549470 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e904e5a-4698-4224-8266-ca42b455a3f0 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:32.554366 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-eb732f5b-0048-42dc-a191-937f3412789b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 23 19:53:32.554366 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:32.554366 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:32.554366 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:32.554366 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:32.554366 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:32.554366 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:32.554366 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:32.554366 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:32.554366 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:32.554366 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:32.554366 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:32.554366 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 19:53:32.554366 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 19:53:32.554366 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 19:53:32.554366 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 19:53:32.555552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:32.555552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:32.555552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 19:53:32.555552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:32.555552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 19:53:32.555552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:32.555552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:32.555552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:32.555552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:32.555552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:32.555552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:32.555552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:32.555552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:32.555552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:32.555552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:32.555552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:32.555552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:32.555552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:32.556713 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:32.556713 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:32.556713 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 19:53:32.556713 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 19:53:32.556713 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 19:53:32.556713 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 19:53:32.556713 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 19:53:32.556713 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 19:53:32.556713 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 23 19:53:32.556713 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:32.559953 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 303/606] 10.222.0.13 () {56 vars in 1117 bytes} [Sun Oct 23 19:53:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:32.560921 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb732f5b-0048-42dc-a191-937f3412789b None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:32.663731 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e904e5a-4698-4224-8266-ca42b455a3f0 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:32.673325 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb732f5b-0048-42dc-a191-937f3412789b None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:32.700297 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c4feb-c24a-4fe7-9e9f-29564c860775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:32.700923 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c4feb-c24a-4fe7-9e9f-29564c860775 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:32.701206 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c4feb-c24a-4fe7-9e9f-29564c860775 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:32.701649 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3c4feb-c24a-4fe7-9e9f-29564c860775 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:32.702058 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c4feb-c24a-4fe7-9e9f-29564c860775 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:32.704017 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3c4feb-c24a-4fe7-9e9f-29564c860775 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:32.709990 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1754b29-7c0e-403e-b3f1-ced917d9a85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:32.710631 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1754b29-7c0e-403e-b3f1-ced917d9a85e None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:32.710912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1754b29-7c0e-403e-b3f1-ced917d9a85e None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:32.711185 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1754b29-7c0e-403e-b3f1-ced917d9a85e None admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:32.711839 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1754b29-7c0e-403e-b3f1-ced917d9a85e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:32.712140 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1754b29-7c0e-403e-b3f1-ced917d9a85e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:32.712968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-4c3c4feb-c24a-4fe7-9e9f-29564c860775 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 23 19:53:32.712968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:32.712968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 19:53:32.712968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 19:53:32.712968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 19:53:32.712968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 19:53:32.712968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 19:53:32.712968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:32.712968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 23 19:53:32.712968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:32.712968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 19:53:32.712968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:32.712968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:32.712968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:32.712968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:32.712968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:32.712968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:32.713965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:32.713965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:32.713965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:32.713965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:32.713965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:32.713965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:32.713965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 19:53:32.713965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 19:53:32.713965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 19:53:32.713965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 19:53:32.713965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:32.713965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:32.713965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:32.713965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:32.713965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:32.713965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:32.719339 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:32.719339 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:32.719339 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:32.719339 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:32.719339 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:32.719339 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:32.719339 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:32.719339 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:32.719339 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:32.719339 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:32.719339 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 19:53:32.719339 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:32.719339 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 23 19:53:32.719339 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:32.720172 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1754b29-7c0e-403e-b3f1-ced917d9a85e None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:32.720257 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 304/607] 10.222.0.13 () {56 vars in 1129 bytes} [Sun Oct 23 19:53:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:32.720257 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c4feb-c24a-4fe7-9e9f-29564c860775 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:32.732122 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 304/608] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:53:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:32.732122 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa6302-8b36-4535-a2d1-5519393a6c4b None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:32.732122 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa6302-8b36-4535-a2d1-5519393a6c4b None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:32.732122 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa6302-8b36-4535-a2d1-5519393a6c4b None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:32.734409 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 305/609] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:32.735203 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa6302-8b36-4535-a2d1-5519393a6c4b None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:32.746410 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:32.800188 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:32.838606 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3c4feb-c24a-4fe7-9e9f-29564c860775 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:32.844921 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa6302-8b36-4535-a2d1-5519393a6c4b None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:32.873114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3338f9b-3c09-4106-a9eb-bdc9792dbb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:32.873745 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3338f9b-3c09-4106-a9eb-bdc9792dbb5a None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:32.874023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3338f9b-3c09-4106-a9eb-bdc9792dbb5a None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:32.874853 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3338f9b-3c09-4106-a9eb-bdc9792dbb5a None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:32.875576 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3338f9b-3c09-4106-a9eb-bdc9792dbb5a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:32.875884 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3338f9b-3c09-4106-a9eb-bdc9792dbb5a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:32.877294 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3338f9b-3c09-4106-a9eb-bdc9792dbb5a None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:32.881938 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03124abc-15fd-4452-8458-374b7d345914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:32.882571 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03124abc-15fd-4452-8458-374b7d345914 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:32.882853 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03124abc-15fd-4452-8458-374b7d345914 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:32.883125 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03124abc-15fd-4452-8458-374b7d345914 None admin] PATH_INFO: `/v3/services` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:32.883755 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03124abc-15fd-4452-8458-374b7d345914 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:32.884063 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03124abc-15fd-4452-8458-374b7d345914 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:32.885442 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03124abc-15fd-4452-8458-374b7d345914 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:32.894586 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 306/610] 10.222.0.13 () {56 vars in 1124 bytes} [Sun Oct 23 19:53:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:32.894586 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03124abc-15fd-4452-8458-374b7d345914 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:32.898905 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 305/611] 10.222.0.13 () {56 vars in 1149 bytes} [Sun Oct 23 19:53:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:32.904569 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52cc7c-fb60-47cf-84be-853978d5b998 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:32.904857 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52cc7c-fb60-47cf-84be-853978d5b998 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:32.905134 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52cc7c-fb60-47cf-84be-853978d5b998 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:32.924195 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-ed52cc7c-fb60-47cf-84be-853978d5b998 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:32.925380 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ed52cc7c-fb60-47cf-84be-853978d5b998 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:33.000387 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03124abc-15fd-4452-8458-374b7d345914 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:33.002277 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 306/612] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:53:33.003318 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52cc7c-fb60-47cf-84be-853978d5b998 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:33.021614 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:33.045872 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:33.057915 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114184d-2810-44cd-8458-aed7da63e885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:33.058655 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114184d-2810-44cd-8458-aed7da63e885 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:33.058942 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114184d-2810-44cd-8458-aed7da63e885 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:33.059215 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5114184d-2810-44cd-8458-aed7da63e885 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:33.074438 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5114184d-2810-44cd-8458-aed7da63e885 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:33.075954 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5114184d-2810-44cd-8458-aed7da63e885 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:33.096461 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-5114184d-2810-44cd-8458-aed7da63e885 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:33.096461 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:33.096461 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:33.096461 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:33.096461 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:33.096461 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:33.096461 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:33.096461 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:33.096461 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:33.096461 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:33.096461 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:33.096461 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:33.096461 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:33.096461 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:33.096461 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:33.096461 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:33.097433 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:33.097433 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:33.097433 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:33.097433 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:33.097433 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:33.097433 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:33.097433 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:33.097433 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:33.097433 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:33.097433 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:33.097433 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:33.097433 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:33.097433 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:33.097433 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:33.097433 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:33.097433 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:33.097433 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:33.098354 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:33.098354 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:33.098354 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:33.102385 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 307/613] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:33.103464 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114184d-2810-44cd-8458-aed7da63e885 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:33.140695 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52cc7c-fb60-47cf-84be-853978d5b998 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:33.187855 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b5480-fc74-40aa-a4ee-f6bc98345ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:33.188472 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b5480-fc74-40aa-a4ee-f6bc98345ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:33.189003 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b5480-fc74-40aa-a4ee-f6bc98345ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:33.189280 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b5480-fc74-40aa-a4ee-f6bc98345ac7 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:33.189917 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b5480-fc74-40aa-a4ee-f6bc98345ac7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:33.190220 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b5480-fc74-40aa-a4ee-f6bc98345ac7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:33.191611 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b5480-fc74-40aa-a4ee-f6bc98345ac7 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:33.197576 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 307/614] 10.222.0.13 () {56 vars in 1124 bytes} [Sun Oct 23 19:53:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:33.203430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b5480-fc74-40aa-a4ee-f6bc98345ac7 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:33.251239 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5114184d-2810-44cd-8458-aed7da63e885 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:33.291727 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:33.295207 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98959b58-39fb-4a2b-b60c-8e7a44ba4b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:33.296481 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98959b58-39fb-4a2b-b60c-8e7a44ba4b5e None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:33.296911 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98959b58-39fb-4a2b-b60c-8e7a44ba4b5e None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:33.297365 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98959b58-39fb-4a2b-b60c-8e7a44ba4b5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:33.329639 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b5480-fc74-40aa-a4ee-f6bc98345ac7 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:33.355962 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:33.373310 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e7c35-9fce-4f2c-9af8-fc16610972fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:33.373928 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e7c35-9fce-4f2c-9af8-fc16610972fe None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:33.374238 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e7c35-9fce-4f2c-9af8-fc16610972fe None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:33.374542 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e7c35-9fce-4f2c-9af8-fc16610972fe None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:33.375183 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36e7c35-9fce-4f2c-9af8-fc16610972fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:33.375487 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36e7c35-9fce-4f2c-9af8-fc16610972fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:33.376841 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36e7c35-9fce-4f2c-9af8-fc16610972fe None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:33.381891 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98959b58-39fb-4a2b-b60c-8e7a44ba4b5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:33.383931 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98959b58-39fb-4a2b-b60c-8e7a44ba4b5e None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:33.393092 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 308/615] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:33.393092 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e7c35-9fce-4f2c-9af8-fc16610972fe None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:33.494392 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e7c35-9fce-4f2c-9af8-fc16610972fe None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:33.496899 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 308/616] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Oct 23 19:53:33 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 23 19:53:33.497753 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98959b58-39fb-4a2b-b60c-8e7a44ba4b5e None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:33.524717 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:33.531380 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491b533-4383-44e0-900f-9c4341fba428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:33.531995 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491b533-4383-44e0-900f-9c4341fba428 None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:33.532277 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491b533-4383-44e0-900f-9c4341fba428 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:33.532538 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9491b533-4383-44e0-900f-9c4341fba428 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:33.533208 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9491b533-4383-44e0-900f-9c4341fba428 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:33.534594 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9491b533-4383-44e0-900f-9c4341fba428 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:33.557431 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 309/617] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 19:53:33 2022] POST /identity/v3/endpoints => generated 346 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:53:33.558316 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491b533-4383-44e0-900f-9c4341fba428 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:33.587662 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:33.614632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98959b58-39fb-4a2b-b60c-8e7a44ba4b5e None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:33.663981 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784a4283-c3cf-4743-89ee-233fb0cbc987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:33.664643 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a4283-c3cf-4743-89ee-233fb0cbc987 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:33.664924 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a4283-c3cf-4743-89ee-233fb0cbc987 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:33.665197 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784a4283-c3cf-4743-89ee-233fb0cbc987 None admin] PATH_INFO: `/v3/services/network` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:33.665796 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784a4283-c3cf-4743-89ee-233fb0cbc987 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:33.667314 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784a4283-c3cf-4743-89ee-233fb0cbc987 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:33.678638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-784a4283-c3cf-4743-89ee-233fb0cbc987 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 23 19:53:33.678638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:33.678638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 19:53:33.678638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 19:53:33.678638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 19:53:33.678638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 19:53:33.678638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 19:53:33.678638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:33.678638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 23 19:53:33.678638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:33.678638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 19:53:33.678638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:33.678638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:33.678638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:33.678638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:33.678638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:33.678638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:33.679934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:33.679934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:33.679934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:33.679934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:33.679934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:33.679934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:33.679934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 19:53:33.679934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 19:53:33.679934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 19:53:33.679934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 19:53:33.679934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:33.679934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:33.679934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:33.679934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:33.679934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:33.679934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:33.681305 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:33.681305 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:33.681305 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:33.681305 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:33.681305 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:33.681305 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:33.681305 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:33.681305 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:33.681305 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:33.681305 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:33.681305 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 19:53:33.681305 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:33.681305 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 23 19:53:33.681305 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:33.681305 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 309/618] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:53:33 2022] GET /identity/v3/services/network => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:33.682118 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c948c-b49d-4b68-8d94-616503435965 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:33.682118 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c948c-b49d-4b68-8d94-616503435965 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:33.682118 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c948c-b49d-4b68-8d94-616503435965 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:33.683402 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 310/619] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:33.683908 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c948c-b49d-4b68-8d94-616503435965 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:33.708906 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9491b533-4383-44e0-900f-9c4341fba428 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:33.752045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f856478b-25ed-45f6-b64e-1ad87e4cf68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:33.752796 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856478b-25ed-45f6-b64e-1ad87e4cf68c None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:33.753077 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856478b-25ed-45f6-b64e-1ad87e4cf68c None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:33.753359 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856478b-25ed-45f6-b64e-1ad87e4cf68c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:33.762636 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f856478b-25ed-45f6-b64e-1ad87e4cf68c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:33.764080 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f856478b-25ed-45f6-b64e-1ad87e4cf68c None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:33.779781 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-f856478b-25ed-45f6-b64e-1ad87e4cf68c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 23 19:53:33.779781 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:33.779781 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:33.779781 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:33.779781 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:33.779781 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:33.779781 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:33.779781 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:33.779781 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:33.779781 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:33.779781 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:33.779781 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:33.779781 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 19:53:33.779781 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 19:53:33.779781 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 19:53:33.779781 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 19:53:33.781430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:33.781430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:33.781430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 19:53:33.781430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:33.781430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 19:53:33.781430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:33.781430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:33.781430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:33.781430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:33.781430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:33.781430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:33.781430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:33.781430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:33.781430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:33.781430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:33.781430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:33.781430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:33.781430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:33.782440 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:33.782440 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:33.782440 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 19:53:33.782440 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 19:53:33.782440 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 19:53:33.782440 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 19:53:33.782440 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 19:53:33.782440 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 19:53:33.782440 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 23 19:53:33.782440 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:33.782440 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 310/620] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Oct 23 19:53:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:33.783042 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1de7d-f5ca-4034-94b3-d0c9f809ce4f None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:33.783042 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1de7d-f5ca-4034-94b3-d0c9f809ce4f None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:33.783176 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f1de7d-f5ca-4034-94b3-d0c9f809ce4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:33.802774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-65f1de7d-f5ca-4034-94b3-d0c9f809ce4f None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:33.804111 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-65f1de7d-f5ca-4034-94b3-d0c9f809ce4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:33.812709 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c948c-b49d-4b68-8d94-616503435965 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:33.847218 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd663be-2735-44f3-b22f-e92f65722b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:33.847771 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd663be-2735-44f3-b22f-e92f65722b1d None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:33.848047 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd663be-2735-44f3-b22f-e92f65722b1d None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:33.848317 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd663be-2735-44f3-b22f-e92f65722b1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:33.855683 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:33.855932 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:33.862325 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd663be-2735-44f3-b22f-e92f65722b1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:33.863926 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd663be-2735-44f3-b22f-e92f65722b1d None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:33.878898 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-3dd663be-2735-44f3-b22f-e92f65722b1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:33.878898 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:33.878898 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:33.878898 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:33.878898 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:33.878898 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:33.878898 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:33.878898 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:33.878898 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:33.878898 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:33.878898 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:33.878898 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:33.878898 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:33.878898 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:33.878898 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:33.878898 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:33.879901 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:33.879901 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:33.879901 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:33.879901 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:33.879901 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:33.879901 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:33.879901 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:33.879901 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:33.879901 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:33.879901 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:33.879901 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:33.879901 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:33.879901 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:33.879901 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:33.879901 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:33.879901 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:33.879901 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:33.880813 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:33.880813 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:33.880813 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:33.880813 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 311/621] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:33.880813 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd663be-2735-44f3-b22f-e92f65722b1d None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:33.887556 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 311/622] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:53:33.888401 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f1de7d-f5ca-4034-94b3-d0c9f809ce4f None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:33.996317 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd663be-2735-44f3-b22f-e92f65722b1d None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:34.005024 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f1de7d-f5ca-4034-94b3-d0c9f809ce4f None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:34.042135 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca2fd5-a0aa-478f-9c1c-36268aec47e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:34.042794 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca2fd5-a0aa-478f-9c1c-36268aec47e0 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:34.043076 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca2fd5-a0aa-478f-9c1c-36268aec47e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:34.043344 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca2fd5-a0aa-478f-9c1c-36268aec47e0 None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:34.044079 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca2fd5-a0aa-478f-9c1c-36268aec47e0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:34.044380 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca2fd5-a0aa-478f-9c1c-36268aec47e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:34.045796 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca2fd5-a0aa-478f-9c1c-36268aec47e0 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:34.047070 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b5953-3059-450f-beaa-209a71acad92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:34.047914 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b5953-3059-450f-beaa-209a71acad92 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:34.048400 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b5953-3059-450f-beaa-209a71acad92 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:34.048880 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736b5953-3059-450f-beaa-209a71acad92 None admin] PATH_INFO: `/v3/services` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:34.049709 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b5953-3059-450f-beaa-209a71acad92 None admin] RBAC: Adding query filter params (name=network) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:34.050309 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b5953-3059-450f-beaa-209a71acad92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:34.051862 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736b5953-3059-450f-beaa-209a71acad92 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:34.057261 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 312/623] 10.222.0.13 () {56 vars in 1112 bytes} [Sun Oct 23 19:53:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:34.058425 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b5953-3059-450f-beaa-209a71acad92 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:34.080184 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 312/624] 10.222.0.13 () {56 vars in 1135 bytes} [Sun Oct 23 19:53:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:34.080184 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca2fd5-a0aa-478f-9c1c-36268aec47e0 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:34.122471 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:34.128417 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:34.178473 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736b5953-3059-450f-beaa-209a71acad92 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:34.198828 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca2fd5-a0aa-478f-9c1c-36268aec47e0 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:34.215665 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f26ff81-0a5c-4166-93f6-8f9522a06d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:34.216614 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26ff81-0a5c-4166-93f6-8f9522a06d11 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:34.217069 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26ff81-0a5c-4166-93f6-8f9522a06d11 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:34.217532 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f26ff81-0a5c-4166-93f6-8f9522a06d11 None admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:34.218420 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f26ff81-0a5c-4166-93f6-8f9522a06d11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:34.219017 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f26ff81-0a5c-4166-93f6-8f9522a06d11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:34.220457 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f26ff81-0a5c-4166-93f6-8f9522a06d11 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:34.232182 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105a9f6-474b-4379-98a9-119e66710567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:34.232783 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105a9f6-474b-4379-98a9-119e66710567 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:34.237509 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105a9f6-474b-4379-98a9-119e66710567 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:34.237644 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 313/625] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:34.237741 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105a9f6-474b-4379-98a9-119e66710567 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:34.237741 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f105a9f6-474b-4379-98a9-119e66710567 None admin] RBAC: Adding query filter params (type=network) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:34.237895 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f26ff81-0a5c-4166-93f6-8f9522a06d11 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:34.237991 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f105a9f6-474b-4379-98a9-119e66710567 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:34.237991 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f105a9f6-474b-4379-98a9-119e66710567 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:34.246764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 313/626] 10.222.0.13 () {56 vars in 1112 bytes} [Sun Oct 23 19:53:34 2022] GET /identity/v3/services?type=network => generated 350 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:34.246764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105a9f6-474b-4379-98a9-119e66710567 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:34.344834 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f26ff81-0a5c-4166-93f6-8f9522a06d11 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:34.345771 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:34.349067 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105a9f6-474b-4379-98a9-119e66710567 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:34.360968 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:34.382099 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ff4fa-3116-4cf2-9b9b-0929008d6298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:34.382674 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ff4fa-3116-4cf2-9b9b-0929008d6298 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:34.382951 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ff4fa-3116-4cf2-9b9b-0929008d6298 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:34.383220 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ff4fa-3116-4cf2-9b9b-0929008d6298 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:34.385740 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d195745-cbfd-4e72-919b-5f07e24319fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:34.386361 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d195745-cbfd-4e72-919b-5f07e24319fd None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:34.386644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d195745-cbfd-4e72-919b-5f07e24319fd None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:34.386910 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d195745-cbfd-4e72-919b-5f07e24319fd None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:34.387557 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d195745-cbfd-4e72-919b-5f07e24319fd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:34.388900 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d195745-cbfd-4e72-919b-5f07e24319fd None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:34.404378 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 314/627] 10.222.0.13 () {60 vars in 1141 bytes} [Sun Oct 23 19:53:34 2022] POST /identity/v3/services => generated 269 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:53:34.406370 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d195745-cbfd-4e72-919b-5f07e24319fd None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:34.467488 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060ff4fa-3116-4cf2-9b9b-0929008d6298 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:34.470911 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060ff4fa-3116-4cf2-9b9b-0929008d6298 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:34.554734 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d195745-cbfd-4e72-919b-5f07e24319fd None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:34.596468 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dcfefc-c98e-4a1e-b542-be0be95fd6a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:34.597513 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dcfefc-c98e-4a1e-b542-be0be95fd6a8 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:34.597990 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dcfefc-c98e-4a1e-b542-be0be95fd6a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:34.598550 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dcfefc-c98e-4a1e-b542-be0be95fd6a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:34.607166 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dcfefc-c98e-4a1e-b542-be0be95fd6a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:34.609222 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dcfefc-c98e-4a1e-b542-be0be95fd6a8 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:34.618062 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 314/628] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Oct 23 19:53:34 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 23 19:53:34.619708 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ff4fa-3116-4cf2-9b9b-0929008d6298 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:34.620855 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-58dcfefc-c98e-4a1e-b542-be0be95fd6a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:34.620855 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:34.620855 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:34.620855 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:34.620855 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:34.620855 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:34.620855 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:34.620855 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:34.620855 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:34.620855 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:34.620855 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:34.620855 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:34.620855 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 19:53:34.620855 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 19:53:34.620855 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 19:53:34.621736 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 19:53:34.621736 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:34.621736 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:34.621736 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:34.621736 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:34.621736 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:34.621736 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:34.621736 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:34.621736 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:34.621736 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:34.621736 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:34.621736 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:34.621736 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:34.621736 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:34.621736 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:34.621736 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:34.621736 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:34.622675 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 19:53:34.622675 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 19:53:34.622675 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 19:53:34.622675 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 19:53:34.622675 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 19:53:34.622675 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 19:53:34.622675 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:34.622675 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:34.622675 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 315/629] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:53:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:34.627394 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dcfefc-c98e-4a1e-b542-be0be95fd6a8 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:34.640690 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:34.651993 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:34.746962 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ff4fa-3116-4cf2-9b9b-0929008d6298 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:34.750033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dcfefc-c98e-4a1e-b542-be0be95fd6a8 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:34.784452 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a3893-6ba5-428b-88db-96c63c8bce4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:34.785305 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a3893-6ba5-428b-88db-96c63c8bce4a None admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:34.785591 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a3893-6ba5-428b-88db-96c63c8bce4a None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:34.785862 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132a3893-6ba5-428b-88db-96c63c8bce4a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:34.786655 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132a3893-6ba5-428b-88db-96c63c8bce4a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:34.787112 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152cb35-accb-4887-a7c6-f2143ec2b067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:34.787982 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152cb35-accb-4887-a7c6-f2143ec2b067 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:34.788099 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132a3893-6ba5-428b-88db-96c63c8bce4a None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:34.788560 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152cb35-accb-4887-a7c6-f2143ec2b067 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:34.789023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5152cb35-accb-4887-a7c6-f2143ec2b067 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:34.800267 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5152cb35-accb-4887-a7c6-f2143ec2b067 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:34.802135 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5152cb35-accb-4887-a7c6-f2143ec2b067 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:34.817922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-5152cb35-accb-4887-a7c6-f2143ec2b067 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:34.817922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:34.817922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:34.817922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:34.817922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:34.817922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:34.817922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:34.817922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:34.817922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:34.817922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:34.817922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:34.817922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:34.817922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:34.817922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:34.817922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:34.817922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:34.819062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:34.819062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:34.819062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:34.819062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:34.819062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:34.819062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:34.819062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:34.819062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:34.819062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:34.819062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:34.819062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:34.819062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:34.819062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:34.819062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:34.819062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:34.819062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:34.819062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:34.819980 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:34.819980 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:34.819980 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:34.820854 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 315/631] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 19:53:34 2022] POST /identity/v3/endpoints => generated 337 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:53:34.822013 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a3893-6ba5-428b-88db-96c63c8bce4a None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:34.825667 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 316/631] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:53:34.825667 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152cb35-accb-4887-a7c6-f2143ec2b067 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:34.903345 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:34.942698 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5152cb35-accb-4887-a7c6-f2143ec2b067 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:34.978897 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:34.989287 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a345444e-80e5-45bc-a186-9f300fdd6715 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:34.990025 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345444e-80e5-45bc-a186-9f300fdd6715 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:34.990350 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345444e-80e5-45bc-a186-9f300fdd6715 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:34.990626 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a345444e-80e5-45bc-a186-9f300fdd6715 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:34.991270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a345444e-80e5-45bc-a186-9f300fdd6715 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:34.991568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a345444e-80e5-45bc-a186-9f300fdd6715 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:34.992958 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a345444e-80e5-45bc-a186-9f300fdd6715 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:35.010970 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 317/632] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:53:35.012297 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a345444e-80e5-45bc-a186-9f300fdd6715 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:35.019967 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132a3893-6ba5-428b-88db-96c63c8bce4a None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:35.090762 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c46837-3c61-4a83-a148-80ef5bb579b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:35.091814 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c46837-3c61-4a83-a148-80ef5bb579b6 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:35.092326 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c46837-3c61-4a83-a148-80ef5bb579b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:35.092801 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c46837-3c61-4a83-a148-80ef5bb579b6 None admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:35.094277 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c46837-3c61-4a83-a148-80ef5bb579b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:35.094802 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c46837-3c61-4a83-a148-80ef5bb579b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:35.096557 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c46837-3c61-4a83-a148-80ef5bb579b6 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:35.114115 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 316/633] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Oct 23 19:53:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:35.121469 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c46837-3c61-4a83-a148-80ef5bb579b6 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:35.125932 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a345444e-80e5-45bc-a186-9f300fdd6715 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:35.151190 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:35.164742 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b42ac70-405b-44f7-8728-f10fde2dd241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:35.165393 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42ac70-405b-44f7-8728-f10fde2dd241 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:35.165738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42ac70-405b-44f7-8728-f10fde2dd241 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:35.166076 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42ac70-405b-44f7-8728-f10fde2dd241 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:35.174419 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b42ac70-405b-44f7-8728-f10fde2dd241 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:35.175942 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b42ac70-405b-44f7-8728-f10fde2dd241 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:35.190436 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-1b42ac70-405b-44f7-8728-f10fde2dd241 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:35.190436 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:35.190436 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:35.190436 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:35.190436 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:35.190436 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:35.190436 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:35.190436 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:35.190436 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:35.190436 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:35.190436 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:35.190436 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:35.190436 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 19:53:35.190436 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 19:53:35.190436 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 19:53:35.191351 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 19:53:35.191351 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:35.191351 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:35.191351 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:35.191351 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:35.191351 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:35.191351 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:35.191351 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:35.191351 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:35.191351 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:35.191351 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:35.191351 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:35.191351 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:35.191351 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:35.191351 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:35.191351 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:35.191351 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:35.192233 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 19:53:35.192233 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 19:53:35.192233 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 19:53:35.192233 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 19:53:35.192233 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 19:53:35.192233 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 19:53:35.192233 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:35.192233 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:35.196265 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 318/634] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:53:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:35.197430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b42ac70-405b-44f7-8728-f10fde2dd241 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:35.265520 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c46837-3c61-4a83-a148-80ef5bb579b6 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:35.307231 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:35.320179 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e2c1a-7212-44dd-b49e-8369141e1bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:35.321125 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e2c1a-7212-44dd-b49e-8369141e1bbc None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:35.321583 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b42ac70-405b-44f7-8728-f10fde2dd241 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:35.321846 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e2c1a-7212-44dd-b49e-8369141e1bbc None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:35.322351 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4e2c1a-7212-44dd-b49e-8369141e1bbc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:35.323564 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4e2c1a-7212-44dd-b49e-8369141e1bbc None admin] RBAC: Adding query filter params (role.id=5e0524ede3ee479c923c7b609a0ca402, scope.project.id=d40947427f5f43e7b6c82e3605737767, user.id=ff5cfe4d31614e2db9059c69116297cc) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:35.325017 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4e2c1a-7212-44dd-b49e-8369141e1bbc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:35.326744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4e2c1a-7212-44dd-b49e-8369141e1bbc None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:35.342276 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 317/635] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Oct 23 19:53:35 2022] GET /identity/v3/role_assignments?user.id=ff5cfe4d31614e2db9059c69116297cc&scope.project.id=d40947427f5f43e7b6c82e3605737767&role.id=5e0524ede3ee479c923c7b609a0ca402 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:35.376999 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f4d19b-9b2b-42d6-b8aa-9d891b962925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:35.377632 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4d19b-9b2b-42d6-b8aa-9d891b962925 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:35.377911 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4d19b-9b2b-42d6-b8aa-9d891b962925 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:35.378259 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f4d19b-9b2b-42d6-b8aa-9d891b962925 None admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:35.378988 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f4d19b-9b2b-42d6-b8aa-9d891b962925 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:35.379336 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f4d19b-9b2b-42d6-b8aa-9d891b962925 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:35.380809 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f4d19b-9b2b-42d6-b8aa-9d891b962925 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:35.399099 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 319/636] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Oct 23 19:53:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:35.405530 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e2c1a-7212-44dd-b49e-8369141e1bbc None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:35.538499 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4e2c1a-7212-44dd-b49e-8369141e1bbc None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:35.575911 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52e0ea-4579-4192-878e-95c9be6471c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:35.576868 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52e0ea-4579-4192-878e-95c9be6471c3 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:35.577961 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52e0ea-4579-4192-878e-95c9be6471c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:35.578488 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52e0ea-4579-4192-878e-95c9be6471c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:35.579426 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a52e0ea-4579-4192-878e-95c9be6471c3 None admin] RBAC: Adding query filter params (role.id=742dfaf63ef74207a2c9213411513925, scope.project.id=d40947427f5f43e7b6c82e3605737767, user.id=15f603d1e7f54751aa79317e2ea00a31) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:35.579926 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a52e0ea-4579-4192-878e-95c9be6471c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:35.581666 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a52e0ea-4579-4192-878e-95c9be6471c3 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:35.594409 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 318/637] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Oct 23 19:53:35 2022] GET /identity/v3/role_assignments?user.id=15f603d1e7f54751aa79317e2ea00a31&scope.project.id=d40947427f5f43e7b6c82e3605737767&role.id=742dfaf63ef74207a2c9213411513925 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:35.834304 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efb6f7-d146-4732-b508-5844c7be47c2 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:35.834825 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efb6f7-d146-4732-b508-5844c7be47c2 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:35.834825 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efb6f7-d146-4732-b508-5844c7be47c2 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:35.836419 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 320/638] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:35.841879 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7e381-8b25-4d05-b34a-6a99ec8d0e34 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:35.842173 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7e381-8b25-4d05-b34a-6a99ec8d0e34 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:35.842485 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7e381-8b25-4d05-b34a-6a99ec8d0e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:35.864765 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-bfd7e381-8b25-4d05-b34a-6a99ec8d0e34 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:35.866202 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-bfd7e381-8b25-4d05-b34a-6a99ec8d0e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:35.924574 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:35.940212 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 319/639] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:35 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:53:36.232232 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64efb6f7-d146-4732-b508-5844c7be47c2 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:36.282505 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:36.343657 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64efb6f7-d146-4732-b508-5844c7be47c2 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:36.379857 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b2a70-6728-4031-90aa-2d9bcb16f048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:36.380494 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b2a70-6728-4031-90aa-2d9bcb16f048 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:36.380778 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b2a70-6728-4031-90aa-2d9bcb16f048 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:36.381051 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4b2a70-6728-4031-90aa-2d9bcb16f048 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:36.381632 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4b2a70-6728-4031-90aa-2d9bcb16f048 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:36.383025 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4b2a70-6728-4031-90aa-2d9bcb16f048 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:36.392636 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-1e4b2a70-6728-4031-90aa-2d9bcb16f048 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 23 19:53:36.392636 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:36.392636 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 19:53:36.392636 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 19:53:36.392636 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 19:53:36.392636 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 19:53:36.392636 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 19:53:36.392636 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:36.392636 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 23 19:53:36.392636 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:36.392636 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 19:53:36.392636 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:36.392636 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:36.392636 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:36.392636 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:36.392636 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:36.392636 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:36.393606 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:36.393606 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:36.393606 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:36.393606 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:36.393606 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:36.393606 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:36.393606 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 19:53:36.393606 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 19:53:36.393606 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 19:53:36.393606 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 19:53:36.393606 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:36.393606 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:36.393606 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:36.393606 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:36.393606 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:36.393606 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:36.394484 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:36.394484 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:36.394484 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:36.394484 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:36.394484 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:36.394484 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:36.394484 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:36.394484 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:36.394484 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:36.394484 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:36.394484 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 19:53:36.394484 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:36.394484 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 23 19:53:36.394484 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:36.394484 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 321/640] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Oct 23 19:53:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:36.400019 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7e381-8b25-4d05-b34a-6a99ec8d0e34 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:36.507096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7e381-8b25-4d05-b34a-6a99ec8d0e34 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:36.545895 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa6eba-9d0e-41e4-b90a-b61e4019526a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:36.546940 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa6eba-9d0e-41e4-b90a-b61e4019526a None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:36.547441 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa6eba-9d0e-41e4-b90a-b61e4019526a None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:36.547927 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa6eba-9d0e-41e4-b90a-b61e4019526a None admin] PATH_INFO: `/v3/services` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:36.548752 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa6eba-9d0e-41e4-b90a-b61e4019526a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:36.549315 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa6eba-9d0e-41e4-b90a-b61e4019526a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:36.550894 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa6eba-9d0e-41e4-b90a-b61e4019526a None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:36.555864 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 320/641] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Oct 23 19:53:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:36.564599 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b2a70-6728-4031-90aa-2d9bcb16f048 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:36.670232 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4b2a70-6728-4031-90aa-2d9bcb16f048 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:36.682084 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:36.711004 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63463754-b748-4cf6-b2c6-427eced1164a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:36.712016 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63463754-b748-4cf6-b2c6-427eced1164a None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:36.712498 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63463754-b748-4cf6-b2c6-427eced1164a None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:36.712985 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63463754-b748-4cf6-b2c6-427eced1164a None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:36.713805 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63463754-b748-4cf6-b2c6-427eced1164a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:36.714335 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63463754-b748-4cf6-b2c6-427eced1164a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:36.715899 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63463754-b748-4cf6-b2c6-427eced1164a None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:36.721608 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 322/642] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Oct 23 19:53:36 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:36.779074 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285781df-5f90-4b02-b980-e57dee565c7f None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:36.779688 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285781df-5f90-4b02-b980-e57dee565c7f None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:36.780168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285781df-5f90-4b02-b980-e57dee565c7f None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:36.782183 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 321/643] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:36.787814 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fef48d-d356-4a25-aacb-5b1781c9e09d None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:36.788122 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fef48d-d356-4a25-aacb-5b1781c9e09d None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:36.788373 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fef48d-d356-4a25-aacb-5b1781c9e09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:36.808296 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-14fef48d-d356-4a25-aacb-5b1781c9e09d None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:36.809580 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-14fef48d-d356-4a25-aacb-5b1781c9e09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:36.887359 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 323/644] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:36 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:53:37.154611 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285781df-5f90-4b02-b980-e57dee565c7f None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:37.208756 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:37.259109 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285781df-5f90-4b02-b980-e57dee565c7f None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:37.294866 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7e5bb-c574-4d9d-a8e7-65c45c084a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:37.295493 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7e5bb-c574-4d9d-a8e7-65c45c084a79 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:37.295772 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7e5bb-c574-4d9d-a8e7-65c45c084a79 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:37.296067 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb7e5bb-c574-4d9d-a8e7-65c45c084a79 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:37.296686 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7e5bb-c574-4d9d-a8e7-65c45c084a79 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:37.298039 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb7e5bb-c574-4d9d-a8e7-65c45c084a79 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:37.307046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-dcb7e5bb-c574-4d9d-a8e7-65c45c084a79 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 23 19:53:37.307046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:37.307046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 19:53:37.307046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 19:53:37.307046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 19:53:37.307046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 19:53:37.307046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 19:53:37.307046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:37.307046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 23 19:53:37.307046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:37.307046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 19:53:37.307046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:37.307046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:37.307046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:37.307046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:37.307046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:37.307999 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:37.307999 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:37.307999 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:37.307999 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:37.307999 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:37.307999 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:37.307999 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:37.307999 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 19:53:37.307999 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 19:53:37.307999 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 19:53:37.307999 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 19:53:37.307999 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:37.307999 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:37.307999 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:37.307999 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:37.307999 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:37.307999 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:37.308894 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:37.308894 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:37.308894 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:37.308894 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:37.308894 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:37.308894 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:37.308894 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:37.308894 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:37.308894 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:37.308894 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:37.308894 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 19:53:37.308894 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:37.308894 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 23 19:53:37.308894 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:37.308894 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 322/645] 10.222.0.13 () {56 vars in 1132 bytes} [Sun Oct 23 19:53:37 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:37.314884 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fef48d-d356-4a25-aacb-5b1781c9e09d None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:37.422587 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fef48d-d356-4a25-aacb-5b1781c9e09d None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:37.424152 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:37.458842 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc93d5-9191-4f35-bda5-036f5a25f678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:37.459465 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc93d5-9191-4f35-bda5-036f5a25f678 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:37.459745 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc93d5-9191-4f35-bda5-036f5a25f678 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:37.460019 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdc93d5-9191-4f35-bda5-036f5a25f678 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:37.460659 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdc93d5-9191-4f35-bda5-036f5a25f678 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:37.460961 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdc93d5-9191-4f35-bda5-036f5a25f678 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:37.462355 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdc93d5-9191-4f35-bda5-036f5a25f678 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:37.467673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 324/646] 10.222.0.13 () {56 vars in 1126 bytes} [Sun Oct 23 19:53:37 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:37.473695 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7e5bb-c574-4d9d-a8e7-65c45c084a79 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:37.581111 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb7e5bb-c574-4d9d-a8e7-65c45c084a79 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:37.601506 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:37.620072 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4d668d-b900-4586-8aed-2ee1be643ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:37.620679 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d668d-b900-4586-8aed-2ee1be643ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:37.620956 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d668d-b900-4586-8aed-2ee1be643ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:37.621227 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4d668d-b900-4586-8aed-2ee1be643ec7 None admin] PATH_INFO: `/v3/services` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:37.621867 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d668d-b900-4586-8aed-2ee1be643ec7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:37.622170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d668d-b900-4586-8aed-2ee1be643ec7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:37.623551 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4d668d-b900-4586-8aed-2ee1be643ec7 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:37.629148 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 323/647] 10.222.0.13 () {56 vars in 1126 bytes} [Sun Oct 23 19:53:37 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:37.634594 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc93d5-9191-4f35-bda5-036f5a25f678 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:37.666873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1513f-f011-485b-a73b-8d267eaa630d None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:37.667444 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1513f-f011-485b-a73b-8d267eaa630d None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:37.667901 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c1513f-f011-485b-a73b-8d267eaa630d None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:37.669418 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 324/648] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 19:53:37.674912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6816594-de62-450b-a29b-c454d48f5f56 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:37.675137 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6816594-de62-450b-a29b-c454d48f5f56 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:37.675421 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6816594-de62-450b-a29b-c454d48f5f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:37.693719 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-f6816594-de62-450b-a29b-c454d48f5f56 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:37.695348 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-f6816594-de62-450b-a29b-c454d48f5f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:37.747720 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdc93d5-9191-4f35-bda5-036f5a25f678 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:37.787597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4444815-1208-4439-a9c7-b99c84835ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:37.788452 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4444815-1208-4439-a9c7-b99c84835ede None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:37.788740 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4444815-1208-4439-a9c7-b99c84835ede None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:37.789044 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4444815-1208-4439-a9c7-b99c84835ede None admin] PATH_INFO: `/v3/endpoints` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:37.789902 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4444815-1208-4439-a9c7-b99c84835ede None admin] RBAC: Adding query filter params (interface=public, service_id=96f7419e66154e598f58c42a5d9a4099, region_id=RegionOne) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:37.790213 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4444815-1208-4439-a9c7-b99c84835ede None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:37.791382 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 325/649] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:37 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 19:53:37.791640 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4444815-1208-4439-a9c7-b99c84835ede None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:37.803423 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 325/650] 10.222.0.13 () {56 vars in 1251 bytes} [Sun Oct 23 19:53:37 2022] GET /identity/v3/endpoints?service_id=96f7419e66154e598f58c42a5d9a4099&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:37.811425 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6816594-de62-450b-a29b-c454d48f5f56 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:37.915497 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a804174-7b46-42bf-842b-9ab8e2e8d321 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:37.916009 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a804174-7b46-42bf-842b-9ab8e2e8d321 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:37.916089 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a804174-7b46-42bf-842b-9ab8e2e8d321 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:37.917509 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 326/651] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 19:53:37.919384 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6816594-de62-450b-a29b-c454d48f5f56 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:37.926215 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e3e9f-6731-4a39-86d5-38905c11ea05 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:37.926215 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e3e9f-6731-4a39-86d5-38905c11ea05 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:37.926215 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e3e9f-6731-4a39-86d5-38905c11ea05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:37.930618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:37.949091 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-863e3e9f-6731-4a39-86d5-38905c11ea05 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:37.950809 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-863e3e9f-6731-4a39-86d5-38905c11ea05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:37.959232 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df115d5f-0415-4076-a27c-9d432c03740d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:37.960343 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df115d5f-0415-4076-a27c-9d432c03740d None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:37.960629 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df115d5f-0415-4076-a27c-9d432c03740d None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:37.960901 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df115d5f-0415-4076-a27c-9d432c03740d None admin] PATH_INFO: `/v3/services` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:37.961508 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df115d5f-0415-4076-a27c-9d432c03740d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:37.963345 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df115d5f-0415-4076-a27c-9d432c03740d None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:37.971477 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 326/652] 10.222.0.13 () {56 vars in 1087 bytes} [Sun Oct 23 19:53:37 2022] GET /identity/v3/services => generated 1029 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 19:53:37.983384 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:38.048835 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 327/653] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:37 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:53:38.071415 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df115d5f-0415-4076-a27c-9d432c03740d None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:38.182733 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df115d5f-0415-4076-a27c-9d432c03740d None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:38.218770 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cbdf50-7464-4548-80fb-c83049bc2cb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:38.219633 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbdf50-7464-4548-80fb-c83049bc2cb5 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:38.220058 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbdf50-7464-4548-80fb-c83049bc2cb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:38.220498 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cbdf50-7464-4548-80fb-c83049bc2cb5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:38.234281 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45cbdf50-7464-4548-80fb-c83049bc2cb5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:53:38.236977 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-45cbdf50-7464-4548-80fb-c83049bc2cb5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 23 19:53:38.236977 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:38.236977 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:38.236977 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:38.236977 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:38.236977 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:38.236977 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:38.236977 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:38.236977 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:38.236977 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:38.236977 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:38.236977 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:38.236977 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:53:38.236977 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:53:38.236977 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:53:38.236977 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:53:38.237946 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:53:38.237946 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:53:38.237946 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:53:38.237946 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:53:38.237946 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:38.237946 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:38.237946 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:38.237946 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:38.237946 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:38.237946 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:38.237946 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:38.237946 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:38.237946 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:38.237946 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:38.237946 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:38.237946 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:38.238830 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:38.238830 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:38.238830 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:38.238830 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:38.238830 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:53:38.238830 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:53:38.238830 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:53:38.238830 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:53:38.238830 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:53:38.238830 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:53:38.238830 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 23 19:53:38.238830 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:38.238830 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 327/654] 10.222.0.13 () {56 vars in 1120 bytes} [Sun Oct 23 19:53:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:38.245782 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e3e9f-6731-4a39-86d5-38905c11ea05 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:38.315368 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cbdf50-7464-4548-80fb-c83049bc2cb5 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:38.351496 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e3e9f-6731-4a39-86d5-38905c11ea05 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:38.356027 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:38.392976 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280fe1be-115e-46ce-9b82-9de02c38ab90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:38.394083 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280fe1be-115e-46ce-9b82-9de02c38ab90 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:38.394585 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280fe1be-115e-46ce-9b82-9de02c38ab90 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:38.395120 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280fe1be-115e-46ce-9b82-9de02c38ab90 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:38.395981 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280fe1be-115e-46ce-9b82-9de02c38ab90 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:38.396480 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280fe1be-115e-46ce-9b82-9de02c38ab90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:38.398172 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280fe1be-115e-46ce-9b82-9de02c38ab90 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:38.407389 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 328/655] 10.222.0.13 () {56 vars in 1115 bytes} [Sun Oct 23 19:53:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:38.413151 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280fe1be-115e-46ce-9b82-9de02c38ab90 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:38.440684 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:38.444586 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cbdf50-7464-4548-80fb-c83049bc2cb5 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:38.491935 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e63e80-42b2-4454-bfbe-ed5e107b461b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:38.492601 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e63e80-42b2-4454-bfbe-ed5e107b461b None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:38.492878 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e63e80-42b2-4454-bfbe-ed5e107b461b None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:38.493146 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e63e80-42b2-4454-bfbe-ed5e107b461b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:38.493977 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e63e80-42b2-4454-bfbe-ed5e107b461b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:38.496274 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e63e80-42b2-4454-bfbe-ed5e107b461b None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:38.507232 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-49e63e80-42b2-4454-bfbe-ed5e107b461b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 19:53:38.507232 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:38.507232 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 19:53:38.507232 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 19:53:38.507232 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 19:53:38.507232 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 19:53:38.507232 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 19:53:38.507232 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:38.507232 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 19:53:38.507232 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:38.507232 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 19:53:38.507232 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:38.507232 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:38.507232 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:38.507232 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:38.507232 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:38.507232 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:38.508299 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:38.508299 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:38.508299 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:38.508299 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:38.508299 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:38.508299 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:38.508299 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 19:53:38.508299 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 19:53:38.508299 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 19:53:38.508299 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 19:53:38.508299 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:38.508299 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:38.508299 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:38.508299 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:38.508299 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:38.508299 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:38.509117 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:38.509117 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:38.509117 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:38.509117 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:38.509117 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:38.509117 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:38.509117 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:38.509117 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:38.509117 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:38.509117 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:38.509117 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 19:53:38.509117 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:38.509117 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 19:53:38.509117 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:38.509117 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 328/656] 10.222.0.13 () {56 vars in 1126 bytes} [Sun Oct 23 19:53:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:53:38.513959 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e63e80-42b2-4454-bfbe-ed5e107b461b None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:38.537711 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280fe1be-115e-46ce-9b82-9de02c38ab90 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:38.573684 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413ca27-c133-4b8f-97e3-5da786a47dd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:38.574126 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413ca27-c133-4b8f-97e3-5da786a47dd7 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:38.574505 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413ca27-c133-4b8f-97e3-5da786a47dd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:38.574779 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413ca27-c133-4b8f-97e3-5da786a47dd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:38.586721 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a413ca27-c133-4b8f-97e3-5da786a47dd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:38.588163 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a413ca27-c133-4b8f-97e3-5da786a47dd7 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:38.604611 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-a413ca27-c133-4b8f-97e3-5da786a47dd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:38.604611 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:38.604611 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:38.604611 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:38.604611 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:38.604611 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:38.604611 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:38.604611 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:38.604611 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:38.604611 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:38.604611 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:38.604611 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:38.604611 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:38.604611 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:38.604611 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:38.604611 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:38.605604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:38.605604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:38.605604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:38.605604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:38.605604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:38.605604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:38.605604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:38.605604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:38.605604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:38.605604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:38.605604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:38.605604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:38.605604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:38.605604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:38.605604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:38.605604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:38.605604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:38.606546 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:38.606546 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:38.606546 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:38.611105 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 329/657] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:53:38.612708 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413ca27-c133-4b8f-97e3-5da786a47dd7 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:38.648061 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:38.653997 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e63e80-42b2-4454-bfbe-ed5e107b461b None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:38.690931 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da720a-5d6d-4c59-ab07-854e27c25151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:38.691651 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da720a-5d6d-4c59-ab07-854e27c25151 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:38.692028 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da720a-5d6d-4c59-ab07-854e27c25151 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:38.692405 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da720a-5d6d-4c59-ab07-854e27c25151 None admin] PATH_INFO: `/v3/services` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:38.693147 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da720a-5d6d-4c59-ab07-854e27c25151 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:38.693608 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da720a-5d6d-4c59-ab07-854e27c25151 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:38.695218 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26da720a-5d6d-4c59-ab07-854e27c25151 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:38.700238 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 329/658] 10.222.0.13 () {56 vars in 1122 bytes} [Sun Oct 23 19:53:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:53:38.704012 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:38.707212 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da720a-5d6d-4c59-ab07-854e27c25151 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:38.736580 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413ca27-c133-4b8f-97e3-5da786a47dd7 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:38.783060 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab8476-a5e7-46bf-92ea-ca3ba6592f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:38.783679 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab8476-a5e7-46bf-92ea-ca3ba6592f8d None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:38.783961 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab8476-a5e7-46bf-92ea-ca3ba6592f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:38.784262 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab8476-a5e7-46bf-92ea-ca3ba6592f8d None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:38.784913 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab8476-a5e7-46bf-92ea-ca3ba6592f8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:38.785217 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab8476-a5e7-46bf-92ea-ca3ba6592f8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:38.786612 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab8476-a5e7-46bf-92ea-ca3ba6592f8d None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:38.799130 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 330/659] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:38.804561 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab8476-a5e7-46bf-92ea-ca3ba6592f8d None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:38.824753 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da720a-5d6d-4c59-ab07-854e27c25151 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:38.871965 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1722083-2e9c-4c74-87ae-d7f4f3e23e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:38.872960 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1722083-2e9c-4c74-87ae-d7f4f3e23e37 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:38.873689 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1722083-2e9c-4c74-87ae-d7f4f3e23e37 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:38.874708 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1722083-2e9c-4c74-87ae-d7f4f3e23e37 None admin] PATH_INFO: `/v3/services` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:38.876074 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1722083-2e9c-4c74-87ae-d7f4f3e23e37 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:38.876849 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1722083-2e9c-4c74-87ae-d7f4f3e23e37 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:38.879597 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1722083-2e9c-4c74-87ae-d7f4f3e23e37 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:38.886527 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 330/660] 10.222.0.13 () {56 vars in 1122 bytes} [Sun Oct 23 19:53:38 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:38.924561 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab8476-a5e7-46bf-92ea-ca3ba6592f8d None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:38.965896 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b5bfa8-2a75-4153-ae49-dcc8352c835f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:38.966501 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5bfa8-2a75-4153-ae49-dcc8352c835f None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:38.966785 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5bfa8-2a75-4153-ae49-dcc8352c835f None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:38.967124 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5bfa8-2a75-4153-ae49-dcc8352c835f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:38.969849 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:38.977183 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b5bfa8-2a75-4153-ae49-dcc8352c835f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:38.978707 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b5bfa8-2a75-4153-ae49-dcc8352c835f None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:38.993940 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-a0b5bfa8-2a75-4153-ae49-dcc8352c835f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 23 19:53:38.993940 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:38.993940 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:38.993940 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:38.993940 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:38.993940 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:38.993940 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:38.993940 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:38.993940 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:38.993940 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:38.993940 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:38.993940 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:38.993940 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 19:53:38.993940 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 19:53:38.993940 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 19:53:38.993940 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 19:53:38.994974 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:38.994974 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:38.994974 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 19:53:38.994974 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:38.994974 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 19:53:38.994974 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:38.994974 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:38.994974 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:38.994974 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:38.994974 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:38.994974 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:38.994974 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:38.994974 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:38.994974 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:38.994974 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:38.994974 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:38.994974 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:38.994974 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:38.995896 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:38.995896 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:38.995896 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 19:53:38.995896 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 19:53:38.995896 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 19:53:38.995896 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 19:53:38.995896 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 19:53:38.995896 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 19:53:38.995896 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 23 19:53:38.995896 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:38.995896 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 331/661] 10.222.0.13 () {56 vars in 1117 bytes} [Sun Oct 23 19:53:38 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:53:39.001702 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1722083-2e9c-4c74-87ae-d7f4f3e23e37 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:39.020582 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8b885-465c-4ca3-b804-c8eaaaf13120 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:39.020868 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8b885-465c-4ca3-b804-c8eaaaf13120 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:39.021144 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f8b885-465c-4ca3-b804-c8eaaaf13120 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:39.023149 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 332/662] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 19:53:39.029701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685d40c-034d-4ffb-879e-2d49e692e4ef None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:39.029987 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685d40c-034d-4ffb-879e-2d49e692e4ef None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:39.030334 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685d40c-034d-4ffb-879e-2d49e692e4ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:39.034444 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:39.052463 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-a685d40c-034d-4ffb-879e-2d49e692e4ef None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:39.054125 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a685d40c-034d-4ffb-879e-2d49e692e4ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:39.120409 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1722083-2e9c-4c74-87ae-d7f4f3e23e37 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:39.134638 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 333/663] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:39 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:53:39.161005 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631b0bc2-aa2a-45ff-a08d-061d34489b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:39.161674 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631b0bc2-aa2a-45ff-a08d-061d34489b55 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:39.162000 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631b0bc2-aa2a-45ff-a08d-061d34489b55 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:39.162297 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631b0bc2-aa2a-45ff-a08d-061d34489b55 None admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:39.163030 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631b0bc2-aa2a-45ff-a08d-061d34489b55 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:39.163389 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631b0bc2-aa2a-45ff-a08d-061d34489b55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:39.164869 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631b0bc2-aa2a-45ff-a08d-061d34489b55 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:39.184080 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 331/664] 10.222.0.13 () {56 vars in 1149 bytes} [Sun Oct 23 19:53:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:39.189934 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685d40c-034d-4ffb-879e-2d49e692e4ef None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:39.296783 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685d40c-034d-4ffb-879e-2d49e692e4ef None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:39.323458 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:39.333281 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69203376-3e9e-4f38-9007-49e842d16b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:39.334132 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69203376-3e9e-4f38-9007-49e842d16b70 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:39.334663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69203376-3e9e-4f38-9007-49e842d16b70 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:39.335003 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69203376-3e9e-4f38-9007-49e842d16b70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:39.345556 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69203376-3e9e-4f38-9007-49e842d16b70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:39.347105 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69203376-3e9e-4f38-9007-49e842d16b70 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:39.362559 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-69203376-3e9e-4f38-9007-49e842d16b70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:39.362559 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:39.362559 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:39.362559 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:39.362559 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:39.362559 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:39.362559 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:39.362559 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:39.362559 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:39.362559 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:39.362559 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:39.362559 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:39.362559 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:39.362559 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:39.362559 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:39.362559 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:39.363637 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:39.363637 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:39.363637 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:39.363637 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:39.363637 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:39.363637 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:39.363637 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:39.363637 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:39.363637 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:39.363637 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:39.363637 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:39.363637 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:39.363637 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:39.363637 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:39.363637 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:39.363637 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:39.363637 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:39.364606 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:39.364606 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:39.364606 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:39.364606 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 334/665] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:39.370946 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631b0bc2-aa2a-45ff-a08d-061d34489b55 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:39.410554 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69203376-3e9e-4f38-9007-49e842d16b70 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:39.464028 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:39.488714 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631b0bc2-aa2a-45ff-a08d-061d34489b55 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:39.528069 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69203376-3e9e-4f38-9007-49e842d16b70 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:39.536298 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cbb6fd-a009-49d6-afe9-419cf2f9962d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:39.537154 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cbb6fd-a009-49d6-afe9-419cf2f9962d None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:39.537650 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cbb6fd-a009-49d6-afe9-419cf2f9962d None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:39.538624 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cbb6fd-a009-49d6-afe9-419cf2f9962d None admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:39.539875 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cbb6fd-a009-49d6-afe9-419cf2f9962d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:39.540638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cbb6fd-a009-49d6-afe9-419cf2f9962d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:39.542503 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cbb6fd-a009-49d6-afe9-419cf2f9962d None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:39.555965 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 332/666] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:39.561306 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cbb6fd-a009-49d6-afe9-419cf2f9962d None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:39.576534 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7c9af-f6ce-469f-ada2-f5b7cd71169d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:39.577139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7c9af-f6ce-469f-ada2-f5b7cd71169d None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:39.577479 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7c9af-f6ce-469f-ada2-f5b7cd71169d None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:39.577750 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae7c9af-f6ce-469f-ada2-f5b7cd71169d None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:39.578431 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae7c9af-f6ce-469f-ada2-f5b7cd71169d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:39.580551 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae7c9af-f6ce-469f-ada2-f5b7cd71169d None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:39.601960 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 335/667] 10.222.0.13 () {60 vars in 1141 bytes} [Sun Oct 23 19:53:39 2022] POST /identity/v3/services => generated 251 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 19:53:39.685578 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cbb6fd-a009-49d6-afe9-419cf2f9962d None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:39.737668 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7878dc5-e0eb-4c39-9efe-534b9f752236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:39.738610 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7878dc5-e0eb-4c39-9efe-534b9f752236 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:39.738922 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7878dc5-e0eb-4c39-9efe-534b9f752236 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:39.739192 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7878dc5-e0eb-4c39-9efe-534b9f752236 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:39.747387 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7878dc5-e0eb-4c39-9efe-534b9f752236 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:39.748852 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7878dc5-e0eb-4c39-9efe-534b9f752236 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:39.752565 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:39.763593 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-d7878dc5-e0eb-4c39-9efe-534b9f752236 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:39.763593 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:39.763593 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:39.763593 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:39.763593 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:39.763593 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:39.763593 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:39.763593 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:39.763593 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:39.763593 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:39.763593 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:39.763593 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:39.763593 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 19:53:39.763593 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 19:53:39.763593 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 19:53:39.764564 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 19:53:39.764564 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:39.764564 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:39.764564 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:39.764564 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:39.764564 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:39.764564 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:39.764564 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:39.764564 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:39.764564 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:39.764564 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:39.764564 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:39.764564 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:39.764564 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:39.764564 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:39.764564 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:39.764564 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:39.765544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 19:53:39.765544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 19:53:39.765544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 19:53:39.765544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 19:53:39.765544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 19:53:39.765544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 19:53:39.765544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:39.765544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:39.765544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 333/668] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:53:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:39.772356 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7c9af-f6ce-469f-ada2-f5b7cd71169d None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:39.811887 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:39.882520 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae7c9af-f6ce-469f-ada2-f5b7cd71169d None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:39.920386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f705c19-86ff-4004-a3ee-30b256450415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:39.921559 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f705c19-86ff-4004-a3ee-30b256450415 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:39.921842 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f705c19-86ff-4004-a3ee-30b256450415 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:39.922117 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f705c19-86ff-4004-a3ee-30b256450415 None admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:39.922850 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f705c19-86ff-4004-a3ee-30b256450415 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:39.923157 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f705c19-86ff-4004-a3ee-30b256450415 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:39.924581 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f705c19-86ff-4004-a3ee-30b256450415 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:39.937171 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 336/669] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Oct 23 19:53:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:39.943258 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7878dc5-e0eb-4c39-9efe-534b9f752236 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:40.048402 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7878dc5-e0eb-4c39-9efe-534b9f752236 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:40.083350 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a64084b-827a-46a8-8855-0553fd54caf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:40.084325 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64084b-827a-46a8-8855-0553fd54caf0 None admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:40.084756 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64084b-827a-46a8-8855-0553fd54caf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:40.085266 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a64084b-827a-46a8-8855-0553fd54caf0 None admin] PATH_INFO: `/v3/projects/d40947427f5f43e7b6c82e3605737767/users/ff5cfe4d31614e2db9059c69116297cc/roles/032a470a172e4fe7ab796c102ceabf46` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:40.087622 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a64084b-827a-46a8-8855-0553fd54caf0 None admin] RBAC: Authorizing `identity:create_grant(project_id=d40947427f5f43e7b6c82e3605737767, user_id=ff5cfe4d31614e2db9059c69116297cc, role_id=032a470a172e4fe7ab796c102ceabf46)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:40.089200 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a64084b-827a-46a8-8855-0553fd54caf0 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:40.105184 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 334/670] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Oct 23 19:53:39 2022] PUT /identity/v3/projects/d40947427f5f43e7b6c82e3605737767/users/ff5cfe4d31614e2db9059c69116297cc/roles/032a470a172e4fe7ab796c102ceabf46 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 19:53:40.249720 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a2fad-02f9-4bbd-a9ff-ee58bba5533a None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:40.250011 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a2fad-02f9-4bbd-a9ff-ee58bba5533a None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:40.250378 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a2fad-02f9-4bbd-a9ff-ee58bba5533a None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:40.251898 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 337/671] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:40.257721 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0215caf-c441-4882-b6c9-673f9d02a07a None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:40.258046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0215caf-c441-4882-b6c9-673f9d02a07a None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:40.258310 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0215caf-c441-4882-b6c9-673f9d02a07a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:40.278171 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-c0215caf-c441-4882-b6c9-673f9d02a07a None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:40.279420 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-c0215caf-c441-4882-b6c9-673f9d02a07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:40.340348 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:40.354297 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 335/672] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:40 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:53:40.622339 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a2fad-02f9-4bbd-a9ff-ee58bba5533a None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:40.722233 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:40.729225 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a2fad-02f9-4bbd-a9ff-ee58bba5533a None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:40.765577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec4c44-0684-47ae-a9f5-d206c34bb1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:40.766210 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec4c44-0684-47ae-a9f5-d206c34bb1cf None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:40.766512 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec4c44-0684-47ae-a9f5-d206c34bb1cf None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:40.766785 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec4c44-0684-47ae-a9f5-d206c34bb1cf None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:40.767368 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec4c44-0684-47ae-a9f5-d206c34bb1cf None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:40.768718 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec4c44-0684-47ae-a9f5-d206c34bb1cf None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:40.777689 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-93ec4c44-0684-47ae-a9f5-d206c34bb1cf None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 23 19:53:40.777689 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:40.777689 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 19:53:40.777689 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 19:53:40.777689 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 19:53:40.777689 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 19:53:40.777689 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 19:53:40.777689 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:40.777689 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 23 19:53:40.777689 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:40.777689 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 19:53:40.777689 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:40.777689 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:40.777689 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:40.777689 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:40.777689 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:40.778669 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:40.778669 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:40.778669 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:40.778669 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:40.778669 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:40.778669 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:40.778669 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:40.778669 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 19:53:40.778669 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 19:53:40.778669 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 19:53:40.778669 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 19:53:40.778669 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:40.778669 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:40.778669 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:40.778669 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:40.778669 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:40.778669 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:40.779540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:40.779540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:40.779540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:40.779540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:40.779540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:40.779540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:40.779540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:40.779540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:40.779540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:40.779540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:40.779540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 19:53:40.779540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:40.779540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 23 19:53:40.779540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:40.779540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 338/673] 10.222.0.13 () {56 vars in 1132 bytes} [Sun Oct 23 19:53:40 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:40.786761 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0215caf-c441-4882-b6c9-673f9d02a07a None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:40.896233 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0215caf-c441-4882-b6c9-673f9d02a07a None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:40.930557 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1bc51-a3dc-4281-8cc0-dc1a147884c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:40.931174 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1bc51-a3dc-4281-8cc0-dc1a147884c8 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:40.931451 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1bc51-a3dc-4281-8cc0-dc1a147884c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:40.931755 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c1bc51-a3dc-4281-8cc0-dc1a147884c8 None admin] PATH_INFO: `/v3/services` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:40.932352 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c1bc51-a3dc-4281-8cc0-dc1a147884c8 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:40.932651 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c1bc51-a3dc-4281-8cc0-dc1a147884c8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:40.934010 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c1bc51-a3dc-4281-8cc0-dc1a147884c8 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:40.940057 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 336/674] 10.222.0.13 () {56 vars in 1126 bytes} [Sun Oct 23 19:53:40 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:40.945792 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec4c44-0684-47ae-a9f5-d206c34bb1cf None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:41.053068 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec4c44-0684-47ae-a9f5-d206c34bb1cf None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:41.090182 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc827809-6a35-4cac-9e1f-5d1a264a9f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:41.090819 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc827809-6a35-4cac-9e1f-5d1a264a9f3e None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:41.091097 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc827809-6a35-4cac-9e1f-5d1a264a9f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:41.091368 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc827809-6a35-4cac-9e1f-5d1a264a9f3e None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:41.091998 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc827809-6a35-4cac-9e1f-5d1a264a9f3e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:41.092299 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc827809-6a35-4cac-9e1f-5d1a264a9f3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:41.094188 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc827809-6a35-4cac-9e1f-5d1a264a9f3e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:41.099589 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 339/675] 10.222.0.13 () {56 vars in 1126 bytes} [Sun Oct 23 19:53:40 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:41.105161 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1bc51-a3dc-4281-8cc0-dc1a147884c8 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:41.169943 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:41.217111 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c1bc51-a3dc-4281-8cc0-dc1a147884c8 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:41.253496 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c0db5-f54a-44b3-9e6a-3525d3965b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:41.254493 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c0db5-f54a-44b3-9e6a-3525d3965b77 None admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:41.254986 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c0db5-f54a-44b3-9e6a-3525d3965b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:41.255472 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13c0db5-f54a-44b3-9e6a-3525d3965b77 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:41.256355 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c0db5-f54a-44b3-9e6a-3525d3965b77 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:41.257912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e13c0db5-f54a-44b3-9e6a-3525d3965b77 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:41.264250 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ebe92-15c3-41df-8221-c03a60356d14 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:41.264540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ebe92-15c3-41df-8221-c03a60356d14 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:41.264895 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ebe92-15c3-41df-8221-c03a60356d14 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:41.269953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 340/676] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 19:53:41.272700 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279f468-8b2b-41f8-857b-b01277d9827a None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:41.273037 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279f468-8b2b-41f8-857b-b01277d9827a None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:41.273610 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279f468-8b2b-41f8-857b-b01277d9827a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:41.284871 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 337/677] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 19:53:41 2022] POST /identity/v3/endpoints => generated 347 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:53:41.293221 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-b279f468-8b2b-41f8-857b-b01277d9827a None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:41.294654 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-b279f468-8b2b-41f8-857b-b01277d9827a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:41.343623 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:41.381279 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 341/678] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:41 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 19:53:41.654239 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c0db5-f54a-44b3-9e6a-3525d3965b77 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:41.765939 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13c0db5-f54a-44b3-9e6a-3525d3965b77 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:41.776926 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:41.805042 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f178b1-5305-49d9-9a69-6aa1956b13a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:41.805718 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f178b1-5305-49d9-9a69-6aa1956b13a9 None admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:41.805924 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f178b1-5305-49d9-9a69-6aa1956b13a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:41.806197 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f178b1-5305-49d9-9a69-6aa1956b13a9 None admin] PATH_INFO: `/v3/services` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:41.806895 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f178b1-5305-49d9-9a69-6aa1956b13a9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:41.808265 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f178b1-5305-49d9-9a69-6aa1956b13a9 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:41.826660 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb06ece-192d-4eb3-89b6-a255e697545b None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:41.827448 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb06ece-192d-4eb3-89b6-a255e697545b None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:41.827935 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb06ece-192d-4eb3-89b6-a255e697545b None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:41.829656 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 338/680] 10.222.0.13 () {60 vars in 1141 bytes} [Sun Oct 23 19:53:41 2022] POST /identity/v3/services => generated 241 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:53:41.829812 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 342/680] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 19:53:41.835478 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8495b02-2944-4cb7-bd71-07c78513b0a8 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:41.836092 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8495b02-2944-4cb7-bd71-07c78513b0a8 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:41.836515 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8495b02-2944-4cb7-bd71-07c78513b0a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:41.858833 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-e8495b02-2944-4cb7-bd71-07c78513b0a8 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:41.860298 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-e8495b02-2944-4cb7-bd71-07c78513b0a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:41.944847 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 339/681] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:41 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:53:42.219908 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb06ece-192d-4eb3-89b6-a255e697545b None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:42.330944 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb06ece-192d-4eb3-89b6-a255e697545b None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:42.374479 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42909e-a48b-4a28-aba3-8e9130588a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:42.375120 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42909e-a48b-4a28-aba3-8e9130588a38 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:42.375399 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42909e-a48b-4a28-aba3-8e9130588a38 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:42.375668 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42909e-a48b-4a28-aba3-8e9130588a38 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:42.376260 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f42909e-a48b-4a28-aba3-8e9130588a38 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:42.377618 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f42909e-a48b-4a28-aba3-8e9130588a38 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:42.386519 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-4f42909e-a48b-4a28-aba3-8e9130588a38 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 23 19:53:42.386519 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:42.386519 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 19:53:42.386519 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 19:53:42.386519 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 19:53:42.386519 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 19:53:42.386519 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 19:53:42.386519 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:42.386519 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 23 19:53:42.386519 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:42.386519 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 19:53:42.386519 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:42.386519 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:42.386519 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:42.386519 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:42.386519 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:42.386519 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:42.387540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:42.387540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:42.387540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:42.387540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:42.387540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:42.387540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:42.387540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 19:53:42.387540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 19:53:42.387540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 19:53:42.387540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 19:53:42.387540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:42.387540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:42.387540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:42.387540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:42.387540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:42.387540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:42.388398 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:42.388398 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:42.388398 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:42.388398 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:42.388398 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:42.388398 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:42.388398 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:42.388398 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:42.388398 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:42.388398 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:42.388398 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 19:53:42.388398 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:42.388398 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 23 19:53:42.388398 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:42.388398 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 343/682] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Oct 23 19:53:42 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:42.396158 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8495b02-2944-4cb7-bd71-07c78513b0a8 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:42.453416 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f798a8-1dac-4385-9ec4-af2bc86ebab4 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:42.453734 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f798a8-1dac-4385-9ec4-af2bc86ebab4 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:42.453964 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f798a8-1dac-4385-9ec4-af2bc86ebab4 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:42.456613 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 344/683] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 19:53:42.460485 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419dcddf-36e5-4dac-8b93-6a66ba016d8c None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:42.460767 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419dcddf-36e5-4dac-8b93-6a66ba016d8c None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:42.461043 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419dcddf-36e5-4dac-8b93-6a66ba016d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:42.479499 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-419dcddf-36e5-4dac-8b93-6a66ba016d8c None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:42.480594 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-419dcddf-36e5-4dac-8b93-6a66ba016d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:42.490212 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:42.502268 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:42.511171 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8495b02-2944-4cb7-bd71-07c78513b0a8 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:42.554946 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236951a1-8972-4e4e-951b-c948cfd0ddea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:42.555956 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236951a1-8972-4e4e-951b-c948cfd0ddea None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:42.556456 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236951a1-8972-4e4e-951b-c948cfd0ddea None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:42.556941 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236951a1-8972-4e4e-951b-c948cfd0ddea None admin] PATH_INFO: `/v3/services` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:42.557777 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236951a1-8972-4e4e-951b-c948cfd0ddea None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:42.558305 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236951a1-8972-4e4e-951b-c948cfd0ddea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:42.559876 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-236951a1-8972-4e4e-951b-c948cfd0ddea None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:42.562467 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 345/684] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 19:53:42.565997 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 340/685] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Oct 23 19:53:42 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:42.571846 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419dcddf-36e5-4dac-8b93-6a66ba016d8c None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:42.673253 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419dcddf-36e5-4dac-8b93-6a66ba016d8c None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:42.707819 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730d28b-3408-491a-b91f-7c1031d2a9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:42.708724 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730d28b-3408-491a-b91f-7c1031d2a9e4 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:42.709231 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730d28b-3408-491a-b91f-7c1031d2a9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:42.709737 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730d28b-3408-491a-b91f-7c1031d2a9e4 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:42.710585 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f730d28b-3408-491a-b91f-7c1031d2a9e4 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:42.711081 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f730d28b-3408-491a-b91f-7c1031d2a9e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:42.712612 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f730d28b-3408-491a-b91f-7c1031d2a9e4 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:42.718455 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 346/686] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Oct 23 19:53:42 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:53:42.724443 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236951a1-8972-4e4e-951b-c948cfd0ddea None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:42.829042 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236951a1-8972-4e4e-951b-c948cfd0ddea None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:42.838659 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730d28b-3408-491a-b91f-7c1031d2a9e4 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:42.850159 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:42.869419 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be1d050-64ea-4f91-81de-5cacf3037387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:42.870034 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1d050-64ea-4f91-81de-5cacf3037387 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:42.870341 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1d050-64ea-4f91-81de-5cacf3037387 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:42.870614 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1d050-64ea-4f91-81de-5cacf3037387 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:42.871344 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be1d050-64ea-4f91-81de-5cacf3037387 None admin] RBAC: Adding query filter params (interface=public, service_id=d7c8dc907cb54d8ca33813cdb7497789, region_id=RegionOne) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:42.871774 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be1d050-64ea-4f91-81de-5cacf3037387 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:42.873716 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be1d050-64ea-4f91-81de-5cacf3037387 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:42.883576 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 341/687] 10.222.0.13 () {56 vars in 1251 bytes} [Sun Oct 23 19:53:42 2022] GET /identity/v3/endpoints?service_id=d7c8dc907cb54d8ca33813cdb7497789&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:42.889400 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be1d050-64ea-4f91-81de-5cacf3037387 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:42.923330 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:42.958601 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730d28b-3408-491a-b91f-7c1031d2a9e4 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:42.996001 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b389d3b-4683-4364-869c-220d04142652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:42.997096 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b389d3b-4683-4364-869c-220d04142652 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:42.997571 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b389d3b-4683-4364-869c-220d04142652 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:42.998037 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b389d3b-4683-4364-869c-220d04142652 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:43.009426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b389d3b-4683-4364-869c-220d04142652 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:53:43.013042 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be1d050-64ea-4f91-81de-5cacf3037387 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:43.015811 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-1b389d3b-4683-4364-869c-220d04142652 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 23 19:53:43.015811 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:43.015811 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:43.015811 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:43.015811 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:43.015811 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:43.015811 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:43.015811 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:43.015811 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:43.015811 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:43.015811 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:43.015811 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:43.015811 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:53:43.015811 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:53:43.015811 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:53:43.015811 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:53:43.016740 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:53:43.016740 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:53:43.016740 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:53:43.016740 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:53:43.016740 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:43.016740 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:43.016740 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:43.016740 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:43.016740 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:43.016740 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:43.016740 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:43.016740 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:43.016740 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:43.016740 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:43.016740 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:43.016740 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:43.017601 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:43.017601 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:43.017601 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:43.017601 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:43.017601 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:53:43.017601 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:53:43.017601 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:53:43.017601 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:53:43.017601 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:53:43.017601 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:53:43.017601 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 23 19:53:43.017601 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:43.024890 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 347/688] 10.222.0.13 () {56 vars in 1120 bytes} [Sun Oct 23 19:53:42 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:53:43.026478 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b389d3b-4683-4364-869c-220d04142652 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:43.052864 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e893b-e1b2-445a-a4d3-c630bf7f2556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:43.053479 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e893b-e1b2-445a-a4d3-c630bf7f2556 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:43.053758 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e893b-e1b2-445a-a4d3-c630bf7f2556 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:43.054034 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370e893b-e1b2-445a-a4d3-c630bf7f2556 None admin] PATH_INFO: `/v3/services` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:43.054940 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370e893b-e1b2-445a-a4d3-c630bf7f2556 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:43.056467 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370e893b-e1b2-445a-a4d3-c630bf7f2556 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:43.070345 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 342/689] 10.222.0.13 () {56 vars in 1087 bytes} [Sun Oct 23 19:53:42 2022] GET /identity/v3/services => generated 1497 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 19:53:43.108222 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:43.148653 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b389d3b-4683-4364-869c-220d04142652 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:43.187501 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088599d0-c2ec-47a8-ac97-6bd0eca6b5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:43.188529 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088599d0-c2ec-47a8-ac97-6bd0eca6b5f6 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:43.189268 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088599d0-c2ec-47a8-ac97-6bd0eca6b5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:43.189735 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088599d0-c2ec-47a8-ac97-6bd0eca6b5f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:43.190634 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088599d0-c2ec-47a8-ac97-6bd0eca6b5f6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:43.191136 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088599d0-c2ec-47a8-ac97-6bd0eca6b5f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:43.192767 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088599d0-c2ec-47a8-ac97-6bd0eca6b5f6 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:43.206387 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 348/690] 10.222.0.13 () {56 vars in 1115 bytes} [Sun Oct 23 19:53:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:43.209097 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e893b-e1b2-445a-a4d3-c630bf7f2556 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:43.309441 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:43.324397 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370e893b-e1b2-445a-a4d3-c630bf7f2556 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:43.359979 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b013969-6e48-436d-ac1d-00a7ead2957a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:43.360937 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b013969-6e48-436d-ac1d-00a7ead2957a None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:43.361357 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b013969-6e48-436d-ac1d-00a7ead2957a None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:43.361769 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b013969-6e48-436d-ac1d-00a7ead2957a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:43.372822 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b013969-6e48-436d-ac1d-00a7ead2957a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:43.375023 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b013969-6e48-436d-ac1d-00a7ead2957a None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:43.389568 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-3b013969-6e48-436d-ac1d-00a7ead2957a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:43.389568 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:43.389568 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:43.389568 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:43.389568 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:43.389568 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:43.389568 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:43.389568 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:43.389568 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:43.389568 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:43.389568 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:43.389568 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:43.389568 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:43.389568 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:43.389568 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:43.389568 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:43.391343 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:43.391343 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:43.391343 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:43.391343 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:43.391343 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:43.391343 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:43.391343 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:43.391343 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:43.391343 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:43.391343 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:43.391343 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:43.391343 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:43.391343 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:43.391343 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:43.391343 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:43.391343 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:43.391343 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:43.392402 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:43.392402 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:43.392402 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:43.398354 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 343/691] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:43.399985 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088599d0-c2ec-47a8-ac97-6bd0eca6b5f6 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:43.512124 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088599d0-c2ec-47a8-ac97-6bd0eca6b5f6 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:43.528913 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e671c5f-5a94-4567-afbc-c8d011d1aea1 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:43.529204 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e671c5f-5a94-4567-afbc-c8d011d1aea1 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:43.529466 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e671c5f-5a94-4567-afbc-c8d011d1aea1 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:43.531153 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 344/692] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:43.536042 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce94eb-c8f6-4e35-ad31-75fafb6f3b82 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:43.536326 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce94eb-c8f6-4e35-ad31-75fafb6f3b82 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:43.536602 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce94eb-c8f6-4e35-ad31-75fafb6f3b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:43.541602 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:43.548634 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc6dc1e-b49d-4d01-a68d-66a84d0af1e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:43.549241 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6dc1e-b49d-4d01-a68d-66a84d0af1e3 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:43.549518 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6dc1e-b49d-4d01-a68d-66a84d0af1e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:43.549785 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc6dc1e-b49d-4d01-a68d-66a84d0af1e3 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:43.550455 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc6dc1e-b49d-4d01-a68d-66a84d0af1e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:43.550755 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc6dc1e-b49d-4d01-a68d-66a84d0af1e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:43.552596 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc6dc1e-b49d-4d01-a68d-66a84d0af1e3 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:43.558352 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-06ce94eb-c8f6-4e35-ad31-75fafb6f3b82 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:43.559608 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-06ce94eb-c8f6-4e35-ad31-75fafb6f3b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:43.566792 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 349/693] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:43.572419 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc6dc1e-b49d-4d01-a68d-66a84d0af1e3 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:43.638802 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 345/694] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 19:53:43.684818 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc6dc1e-b49d-4d01-a68d-66a84d0af1e3 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:43.719377 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acca01d-ac1c-4a8d-8c0a-179278ea51cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:43.720010 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acca01d-ac1c-4a8d-8c0a-179278ea51cf None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:43.720286 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acca01d-ac1c-4a8d-8c0a-179278ea51cf None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:43.720557 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acca01d-ac1c-4a8d-8c0a-179278ea51cf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:43.728241 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acca01d-ac1c-4a8d-8c0a-179278ea51cf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:43.729733 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acca01d-ac1c-4a8d-8c0a-179278ea51cf None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:43.742806 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-3acca01d-ac1c-4a8d-8c0a-179278ea51cf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 23 19:53:43.742806 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:43.742806 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:43.742806 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:43.742806 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:43.742806 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:43.742806 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:43.742806 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:43.742806 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:43.742806 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:43.742806 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:43.742806 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:43.742806 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 19:53:43.742806 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 19:53:43.742806 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 19:53:43.742806 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 19:53:43.743754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:43.743754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:43.743754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 19:53:43.743754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:43.743754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 19:53:43.743754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:43.743754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:43.743754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:43.743754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:43.743754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:43.743754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:43.743754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:43.743754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:43.743754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:43.743754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:43.743754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:43.743754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:43.743754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:43.745269 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:43.745269 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:43.745269 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 19:53:43.745269 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 19:53:43.745269 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 19:53:43.745269 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 19:53:43.745269 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 19:53:43.745269 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 19:53:43.745269 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 23 19:53:43.745269 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:43.745269 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 350/695] 10.222.0.13 () {56 vars in 1117 bytes} [Sun Oct 23 19:53:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:53:43.749809 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce94eb-c8f6-4e35-ad31-75fafb6f3b82 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:43.819134 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:43.850087 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce94eb-c8f6-4e35-ad31-75fafb6f3b82 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:43.884142 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25002d-3c23-429e-af74-395d855481fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:43.884750 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25002d-3c23-429e-af74-395d855481fd None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:43.885095 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25002d-3c23-429e-af74-395d855481fd None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:43.885352 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb25002d-3c23-429e-af74-395d855481fd None admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:43.886091 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25002d-3c23-429e-af74-395d855481fd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:43.886422 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25002d-3c23-429e-af74-395d855481fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:43.887855 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb25002d-3c23-429e-af74-395d855481fd None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:43.907472 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 346/696] 10.222.0.13 () {56 vars in 1149 bytes} [Sun Oct 23 19:53:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:43.912693 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acca01d-ac1c-4a8d-8c0a-179278ea51cf None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:43.952241 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25002d-3c23-429e-af74-395d855481fd None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:43.966768 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:44.038793 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acca01d-ac1c-4a8d-8c0a-179278ea51cf None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:44.062381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb25002d-3c23-429e-af74-395d855481fd None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:44.077997 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d716f53-12bf-4c93-886c-effbb6c4ea53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:44.078783 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d716f53-12bf-4c93-886c-effbb6c4ea53 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:44.079186 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d716f53-12bf-4c93-886c-effbb6c4ea53 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:44.079467 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d716f53-12bf-4c93-886c-effbb6c4ea53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:44.091313 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:44.100661 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fadf32f-6380-4c28-9908-55fdd0f46e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:44.101304 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fadf32f-6380-4c28-9908-55fdd0f46e50 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:44.101663 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fadf32f-6380-4c28-9908-55fdd0f46e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:44.101868 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fadf32f-6380-4c28-9908-55fdd0f46e50 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:44.102457 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fadf32f-6380-4c28-9908-55fdd0f46e50 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:44.103896 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fadf32f-6380-4c28-9908-55fdd0f46e50 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:44.113119 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-9fadf32f-6380-4c28-9908-55fdd0f46e50 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 23 19:53:44.113119 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:44.113119 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 19:53:44.113119 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 19:53:44.113119 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 19:53:44.113119 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 19:53:44.113119 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 19:53:44.113119 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:44.113119 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 23 19:53:44.113119 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:44.113119 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 19:53:44.113119 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:44.113119 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:44.113119 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:44.113119 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:44.113119 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:44.113119 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:44.114179 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:44.114179 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:44.114179 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:44.114179 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:44.114179 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:44.114179 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:44.114179 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 19:53:44.114179 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 19:53:44.114179 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 19:53:44.114179 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 19:53:44.114179 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:44.114179 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:44.114179 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:44.114179 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:44.114179 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:44.114179 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:44.115786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:44.115786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:44.115786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:44.115786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:44.115786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:44.115786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:44.115786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:44.115786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:44.115786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:44.115786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:44.115786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 19:53:44.115786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:44.115786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 23 19:53:44.115786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:44.115786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 347/697] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:53:43 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:53:44.120754 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417bb64-fa5c-4809-b8ee-3c8da14aa38f None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:44.121135 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417bb64-fa5c-4809-b8ee-3c8da14aa38f None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:44.121301 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1417bb64-fa5c-4809-b8ee-3c8da14aa38f None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:44.122881 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 348/698] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:44.123465 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417bb64-fa5c-4809-b8ee-3c8da14aa38f None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:44.163675 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d716f53-12bf-4c93-886c-effbb6c4ea53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:44.165042 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d716f53-12bf-4c93-886c-effbb6c4ea53 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:44.216085 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:44.231164 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1417bb64-fa5c-4809-b8ee-3c8da14aa38f None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:44.266165 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91e37b-b3a3-4837-b957-086d95b5daea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:44.266830 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91e37b-b3a3-4837-b957-086d95b5daea None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:44.267109 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91e37b-b3a3-4837-b957-086d95b5daea None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:44.267426 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd91e37b-b3a3-4837-b957-086d95b5daea None admin] PATH_INFO: `/v3/services` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:44.268101 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd91e37b-b3a3-4837-b957-086d95b5daea None admin] RBAC: Adding query filter params (name=compute) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:44.268405 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd91e37b-b3a3-4837-b957-086d95b5daea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:44.269156 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 351/699] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Oct 23 19:53:43 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 23 19:53:44.270177 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c9997-2973-4226-85a6-f67cebde88d1 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:44.270488 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c9997-2973-4226-85a6-f67cebde88d1 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:44.270761 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c9997-2973-4226-85a6-f67cebde88d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:44.274732 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd91e37b-b3a3-4837-b957-086d95b5daea None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:44.284220 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 349/700] 10.222.0.13 () {56 vars in 1112 bytes} [Sun Oct 23 19:53:44 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:44.284220 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91e37b-b3a3-4837-b957-086d95b5daea None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:44.291659 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-b26c9997-2973-4226-85a6-f67cebde88d1 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:44.293241 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-b26c9997-2973-4226-85a6-f67cebde88d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:44.342176 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:44.364268 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 352/701] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:53:44.364997 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c9997-2973-4226-85a6-f67cebde88d1 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:44.402338 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd91e37b-b3a3-4837-b957-086d95b5daea None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:44.448106 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c6595-7bef-4c42-92fd-5db95c56a73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:44.449472 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c6595-7bef-4c42-92fd-5db95c56a73a None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:44.449749 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c6595-7bef-4c42-92fd-5db95c56a73a None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:44.450021 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c6595-7bef-4c42-92fd-5db95c56a73a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:44.460897 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:44.465726 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c6595-7bef-4c42-92fd-5db95c56a73a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:44.468492 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c6595-7bef-4c42-92fd-5db95c56a73a None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:44.482589 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c9997-2973-4226-85a6-f67cebde88d1 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:44.493974 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-a44c6595-7bef-4c42-92fd-5db95c56a73a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:44.493974 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:44.493974 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:44.493974 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:44.493974 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:44.493974 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:44.493974 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:44.493974 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:44.493974 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:44.493974 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:44.493974 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:44.493974 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:44.493974 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:44.493974 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:44.493974 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:44.493974 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:44.495244 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:44.495244 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:44.495244 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:44.495244 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:44.495244 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:44.495244 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:44.495244 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:44.495244 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:44.495244 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:44.495244 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:44.495244 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:44.495244 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:44.495244 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:44.495244 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:44.495244 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:44.495244 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:44.495244 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:44.496310 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:44.496310 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:44.496310 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:44.498429 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 350/702] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:44.503980 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c6595-7bef-4c42-92fd-5db95c56a73a None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:44.528506 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23006c2-f8fe-4b48-a501-3ec018cda0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:44.529139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23006c2-f8fe-4b48-a501-3ec018cda0df None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:44.529418 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23006c2-f8fe-4b48-a501-3ec018cda0df None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:44.529693 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23006c2-f8fe-4b48-a501-3ec018cda0df None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:44.530401 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23006c2-f8fe-4b48-a501-3ec018cda0df None admin] RBAC: Adding query filter params (type=compute) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:44.530704 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23006c2-f8fe-4b48-a501-3ec018cda0df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:44.532302 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23006c2-f8fe-4b48-a501-3ec018cda0df None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:44.542353 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 353/703] 10.222.0.13 () {56 vars in 1112 bytes} [Sun Oct 23 19:53:44 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:44.618208 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c6595-7bef-4c42-92fd-5db95c56a73a None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:44.628791 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:44.635450 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23006c2-f8fe-4b48-a501-3ec018cda0df None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:44.670038 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1f37e-bc12-4da0-a9ad-7fc3fa669c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:44.670704 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1f37e-bc12-4da0-a9ad-7fc3fa669c8e None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:44.671027 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1f37e-bc12-4da0-a9ad-7fc3fa669c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:44.671302 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f1f37e-bc12-4da0-a9ad-7fc3fa669c8e None admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:44.672254 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1f37e-bc12-4da0-a9ad-7fc3fa669c8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:44.672560 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1f37e-bc12-4da0-a9ad-7fc3fa669c8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:44.674770 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f1f37e-bc12-4da0-a9ad-7fc3fa669c8e None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:44.689642 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 351/704] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:53:44.695624 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1f37e-bc12-4da0-a9ad-7fc3fa669c8e None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:44.774187 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23006c2-f8fe-4b48-a501-3ec018cda0df None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:44.811108 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f1f37e-bc12-4da0-a9ad-7fc3fa669c8e None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:44.815228 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731ecc5b-06a7-4925-868f-8d0e0d7ba221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:44.815584 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ecc5b-06a7-4925-868f-8d0e0d7ba221 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:44.815862 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ecc5b-06a7-4925-868f-8d0e0d7ba221 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:44.816132 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ecc5b-06a7-4925-868f-8d0e0d7ba221 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:44.816714 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731ecc5b-06a7-4925-868f-8d0e0d7ba221 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:44.818073 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731ecc5b-06a7-4925-868f-8d0e0d7ba221 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:44.828514 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-731ecc5b-06a7-4925-868f-8d0e0d7ba221 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 19:53:44.828514 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:44.828514 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 19:53:44.828514 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 19:53:44.828514 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 19:53:44.828514 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 19:53:44.828514 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 19:53:44.828514 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:44.828514 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 19:53:44.828514 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:44.828514 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 19:53:44.828514 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:44.828514 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:44.828514 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:44.828514 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:44.828514 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:44.828514 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:44.829550 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:44.829550 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:44.829550 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:44.829550 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:44.829550 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:44.829550 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:44.829550 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 19:53:44.829550 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 19:53:44.829550 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 19:53:44.829550 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 19:53:44.829550 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:44.829550 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:44.829550 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:44.829550 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:44.829550 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:44.829550 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:44.830432 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:44.830432 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:44.830432 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:44.830432 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:44.830432 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:44.830432 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:44.830432 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:44.830432 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:44.830432 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:44.830432 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:44.830432 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 19:53:44.830432 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:44.830432 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 19:53:44.830432 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:44.830432 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 354/705] 10.222.0.13 () {56 vars in 1126 bytes} [Sun Oct 23 19:53:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:53:44.835297 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731ecc5b-06a7-4925-868f-8d0e0d7ba221 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:44.851227 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:44.858581 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a2f9c-1644-474a-a49e-365a177af6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:44.859218 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a2f9c-1644-474a-a49e-365a177af6a7 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:44.859496 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a2f9c-1644-474a-a49e-365a177af6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:44.859763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139a2f9c-1644-474a-a49e-365a177af6a7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:44.868671 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139a2f9c-1644-474a-a49e-365a177af6a7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:44.870786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139a2f9c-1644-474a-a49e-365a177af6a7 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:44.886730 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-139a2f9c-1644-474a-a49e-365a177af6a7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:44.886730 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:44.886730 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:44.886730 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:44.886730 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:44.886730 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:44.886730 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:44.886730 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:44.886730 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:44.886730 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:44.886730 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:44.886730 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:44.886730 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 19:53:44.886730 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 19:53:44.886730 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 19:53:44.887677 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 19:53:44.887677 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:44.887677 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:44.887677 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:44.887677 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:44.887677 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:44.887677 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:44.887677 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:44.887677 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:44.887677 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:44.887677 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:44.887677 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:44.887677 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:44.887677 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:44.887677 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:44.887677 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:44.887677 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:44.889173 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 19:53:44.889173 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 19:53:44.889173 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 19:53:44.889173 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 19:53:44.889173 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 19:53:44.889173 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 19:53:44.889173 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:44.889173 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:44.889173 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 352/706] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:53:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:53:44.893678 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a2f9c-1644-474a-a49e-365a177af6a7 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:44.933697 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:44.964233 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731ecc5b-06a7-4925-868f-8d0e0d7ba221 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:45.022273 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a5799-731f-47dd-8177-fff5a993b927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:45.022273 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a5799-731f-47dd-8177-fff5a993b927 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:45.023070 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139a2f9c-1644-474a-a49e-365a177af6a7 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:45.023152 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a5799-731f-47dd-8177-fff5a993b927 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:45.023152 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7a5799-731f-47dd-8177-fff5a993b927 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:45.023152 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a5799-731f-47dd-8177-fff5a993b927 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:45.023152 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a5799-731f-47dd-8177-fff5a993b927 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:45.023974 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7a5799-731f-47dd-8177-fff5a993b927 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:45.033971 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 355/707] 10.222.0.13 () {56 vars in 1122 bytes} [Sun Oct 23 19:53:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:53:45.034858 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a5799-731f-47dd-8177-fff5a993b927 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:45.057593 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac32232-f601-4561-ac76-61d1da137091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:45.058208 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac32232-f601-4561-ac76-61d1da137091 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:45.058512 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac32232-f601-4561-ac76-61d1da137091 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:45.058781 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac32232-f601-4561-ac76-61d1da137091 None admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:45.059474 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac32232-f601-4561-ac76-61d1da137091 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:45.059782 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac32232-f601-4561-ac76-61d1da137091 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:45.061202 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac32232-f601-4561-ac76-61d1da137091 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:45.074487 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 353/708] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Oct 23 19:53:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:53:45.080512 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac32232-f601-4561-ac76-61d1da137091 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:45.117236 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:45.156454 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7a5799-731f-47dd-8177-fff5a993b927 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:45.190259 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:45.200218 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d396d-660f-4103-bc6b-37f1765ed4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:45.201600 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d396d-660f-4103-bc6b-37f1765ed4c7 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:45.202038 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d396d-660f-4103-bc6b-37f1765ed4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:45.203018 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0d396d-660f-4103-bc6b-37f1765ed4c7 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:45.203801 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac32232-f601-4561-ac76-61d1da137091 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:45.203964 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d396d-660f-4103-bc6b-37f1765ed4c7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:45.204403 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d396d-660f-4103-bc6b-37f1765ed4c7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:45.205907 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0d396d-660f-4103-bc6b-37f1765ed4c7 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:45.216259 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 356/709] 10.222.0.13 () {56 vars in 1122 bytes} [Sun Oct 23 19:53:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:45.217372 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d396d-660f-4103-bc6b-37f1765ed4c7 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:45.240392 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c0565d-d5d9-4858-909a-cb5703bf10d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:45.241006 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0565d-d5d9-4858-909a-cb5703bf10d0 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:45.241551 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0565d-d5d9-4858-909a-cb5703bf10d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:45.241672 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c0565d-d5d9-4858-909a-cb5703bf10d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:45.242405 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c0565d-d5d9-4858-909a-cb5703bf10d0 None admin] RBAC: Adding query filter params (role.id=032a470a172e4fe7ab796c102ceabf46, scope.project.id=d40947427f5f43e7b6c82e3605737767, user.id=ff5cfe4d31614e2db9059c69116297cc) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:45.242720 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c0565d-d5d9-4858-909a-cb5703bf10d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:45.244149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c0565d-d5d9-4858-909a-cb5703bf10d0 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:45.258400 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 354/710] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Oct 23 19:53:45 2022] GET /identity/v3/role_assignments?user.id=ff5cfe4d31614e2db9059c69116297cc&scope.project.id=d40947427f5f43e7b6c82e3605737767&role.id=032a470a172e4fe7ab796c102ceabf46 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:45.339124 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0d396d-660f-4103-bc6b-37f1765ed4c7 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:45.379830 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6aef6a-e59e-4391-bb61-e65728fc56f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:45.380528 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6aef6a-e59e-4391-bb61-e65728fc56f2 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:45.380819 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6aef6a-e59e-4391-bb61-e65728fc56f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:45.381099 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6aef6a-e59e-4391-bb61-e65728fc56f2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:45.381820 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6aef6a-e59e-4391-bb61-e65728fc56f2 None admin] RBAC: Adding query filter params (interface=public, service_id=49c001dad7a5485684b3fd87e3ffc6f7, region_id=RegionOne) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:45.382128 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6aef6a-e59e-4391-bb61-e65728fc56f2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:45.383559 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6aef6a-e59e-4391-bb61-e65728fc56f2 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:45.386630 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:45.394717 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 357/711] 10.222.0.13 () {56 vars in 1251 bytes} [Sun Oct 23 19:53:45 2022] GET /identity/v3/endpoints?service_id=49c001dad7a5485684b3fd87e3ffc6f7&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:45.396664 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c0565d-d5d9-4858-909a-cb5703bf10d0 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:45.458515 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd3f3f-7d3d-430f-b58e-00c4c6b8480c None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:45.458855 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd3f3f-7d3d-430f-b58e-00c4c6b8480c None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:45.459077 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd3f3f-7d3d-430f-b58e-00c4c6b8480c None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:45.460688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 358/712] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 19:53:45.466725 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a87870-7991-4993-af88-24425fb5dbf4 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:45.467298 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a87870-7991-4993-af88-24425fb5dbf4 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:45.468237 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a87870-7991-4993-af88-24425fb5dbf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:45.494851 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-94a87870-7991-4993-af88-24425fb5dbf4 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:45.495549 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-94a87870-7991-4993-af88-24425fb5dbf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:45.515423 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c0565d-d5d9-4858-909a-cb5703bf10d0 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:45.554528 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b1215-e93f-4d46-b42c-add930f67b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:45.555138 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b1215-e93f-4d46-b42c-add930f67b71 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:45.555691 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b1215-e93f-4d46-b42c-add930f67b71 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:45.555691 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536b1215-e93f-4d46-b42c-add930f67b71 None admin] PATH_INFO: `/v3/services` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:45.556274 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b1215-e93f-4d46-b42c-add930f67b71 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:45.557651 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536b1215-e93f-4d46-b42c-add930f67b71 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:45.565078 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 355/713] 10.222.0.13 () {56 vars in 1087 bytes} [Sun Oct 23 19:53:45 2022] GET /identity/v3/services => generated 1497 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 19:53:45.591760 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 359/714] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:45 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:53:45.859455 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b1215-e93f-4d46-b42c-add930f67b71 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:45.880136 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:45.964237 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536b1215-e93f-4d46-b42c-add930f67b71 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:45.999661 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf341f-1048-4cd4-84e4-caf99fb47d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:46.001664 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf341f-1048-4cd4-84e4-caf99fb47d7e None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:46.002102 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf341f-1048-4cd4-84e4-caf99fb47d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:46.002543 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccf341f-1048-4cd4-84e4-caf99fb47d7e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:46.003270 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf341f-1048-4cd4-84e4-caf99fb47d7e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:46.004796 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccf341f-1048-4cd4-84e4-caf99fb47d7e None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:46.014121 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-6ccf341f-1048-4cd4-84e4-caf99fb47d7e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 23 19:53:46.014121 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:46.014121 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 19:53:46.014121 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 19:53:46.014121 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 19:53:46.014121 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 19:53:46.014121 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 19:53:46.014121 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:46.014121 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 23 19:53:46.014121 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:46.014121 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 19:53:46.014121 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:46.014121 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:46.014121 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:46.014121 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:46.014121 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:46.014121 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:46.015454 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:46.015454 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:46.015454 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:46.015454 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:46.015454 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:46.015454 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:46.015454 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 19:53:46.015454 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 19:53:46.015454 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 19:53:46.015454 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 19:53:46.015454 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:46.015454 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:46.015454 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:46.015454 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:46.015454 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:46.015454 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:46.016460 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:46.016460 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:46.016460 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:46.016460 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:46.016460 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:46.016460 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:46.016460 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:46.016460 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:46.016460 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:46.016460 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:46.016460 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 19:53:46.016460 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:46.016460 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 23 19:53:46.016460 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:46.018661 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 356/715] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Oct 23 19:53:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:46.024526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a87870-7991-4993-af88-24425fb5dbf4 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:46.130789 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a87870-7991-4993-af88-24425fb5dbf4 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:46.161219 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:46.167688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17edd0a-0da4-403d-ae8d-bd9969ccc330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:46.168364 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17edd0a-0da4-403d-ae8d-bd9969ccc330 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:46.168830 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17edd0a-0da4-403d-ae8d-bd9969ccc330 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:46.169109 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17edd0a-0da4-403d-ae8d-bd9969ccc330 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:46.169747 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17edd0a-0da4-403d-ae8d-bd9969ccc330 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:46.170052 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17edd0a-0da4-403d-ae8d-bd9969ccc330 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:46.171522 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17edd0a-0da4-403d-ae8d-bd9969ccc330 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:46.176710 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 360/716] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Oct 23 19:53:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:46.181971 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf341f-1048-4cd4-84e4-caf99fb47d7e None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:46.271750 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:46.292335 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccf341f-1048-4cd4-84e4-caf99fb47d7e None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:46.327323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9655f98c-3ddc-4f51-8c69-f9d35bf15dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:46.327932 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655f98c-3ddc-4f51-8c69-f9d35bf15dae None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:46.328207 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655f98c-3ddc-4f51-8c69-f9d35bf15dae None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:46.328473 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9655f98c-3ddc-4f51-8c69-f9d35bf15dae None admin] PATH_INFO: `/v3/services` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:46.329102 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9655f98c-3ddc-4f51-8c69-f9d35bf15dae None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:46.329396 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9655f98c-3ddc-4f51-8c69-f9d35bf15dae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:46.330776 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9655f98c-3ddc-4f51-8c69-f9d35bf15dae None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:46.336000 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 357/717] 10.222.0.13 () {56 vars in 1114 bytes} [Sun Oct 23 19:53:46 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:46.344093 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17edd0a-0da4-403d-ae8d-bd9969ccc330 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:46.449721 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17edd0a-0da4-403d-ae8d-bd9969ccc330 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:46.487448 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba062165-3e20-49b1-9ab2-f83c4046046c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:46.488891 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba062165-3e20-49b1-9ab2-f83c4046046c None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:46.489372 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba062165-3e20-49b1-9ab2-f83c4046046c None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:46.489855 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba062165-3e20-49b1-9ab2-f83c4046046c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:46.490806 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba062165-3e20-49b1-9ab2-f83c4046046c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:46.492401 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba062165-3e20-49b1-9ab2-f83c4046046c None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:46.518276 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 361/718] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 19:53:46 2022] POST /identity/v3/endpoints => generated 346 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:53:46.800676 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1587c8-c61f-4fb8-a6c1-e1b3c295e27f None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:46.800976 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1587c8-c61f-4fb8-a6c1-e1b3c295e27f None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:46.801251 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1587c8-c61f-4fb8-a6c1-e1b3c295e27f None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:46.802888 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 358/719] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:46.808096 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd79dba-4190-4204-b982-e973a7cff5c3 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:46.808390 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd79dba-4190-4204-b982-e973a7cff5c3 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:46.808675 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd79dba-4190-4204-b982-e973a7cff5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:46.828023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-4bd79dba-4190-4204-b982-e973a7cff5c3 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:46.829219 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4bd79dba-4190-4204-b982-e973a7cff5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:46.877521 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:46.907510 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 362/720] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:53:47.186186 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1587c8-c61f-4fb8-a6c1-e1b3c295e27f None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:47.291736 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1587c8-c61f-4fb8-a6c1-e1b3c295e27f None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:47.326316 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8775d259-1c8c-48d0-b96a-0bf6baf6d91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:47.326931 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775d259-1c8c-48d0-b96a-0bf6baf6d91a None admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:47.327736 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775d259-1c8c-48d0-b96a-0bf6baf6d91a None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:47.328005 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8775d259-1c8c-48d0-b96a-0bf6baf6d91a None admin] PATH_INFO: `/v3/services` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:47.328668 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8775d259-1c8c-48d0-b96a-0bf6baf6d91a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:47.330020 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8775d259-1c8c-48d0-b96a-0bf6baf6d91a None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:47.347054 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 359/721] 10.222.0.13 () {60 vars in 1141 bytes} [Sun Oct 23 19:53:47 2022] POST /identity/v3/services => generated 242 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:53:47.607263 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aee121-5e4e-4ec7-8053-7cf32022aaff None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:47.607782 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aee121-5e4e-4ec7-8053-7cf32022aaff None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:47.607861 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aee121-5e4e-4ec7-8053-7cf32022aaff None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:47.609976 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 363/722] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:47.615356 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193a93c-056b-4f2d-acc9-8ae9bdc16cb5 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:47.615646 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193a93c-056b-4f2d-acc9-8ae9bdc16cb5 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:47.615921 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193a93c-056b-4f2d-acc9-8ae9bdc16cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:47.623188 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:47.634737 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-5193a93c-056b-4f2d-acc9-8ae9bdc16cb5 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:47.636352 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-5193a93c-056b-4f2d-acc9-8ae9bdc16cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:47.717465 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 360/723] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:47 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:53:47.770873 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a5e0c-5d35-41b3-86b4-e5a1c2f8d29d None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:47.771189 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a5e0c-5d35-41b3-86b4-e5a1c2f8d29d None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:47.771433 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9a5e0c-5d35-41b3-86b4-e5a1c2f8d29d None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:47.773055 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 364/724] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:47.778268 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803cfc4-6abe-4800-a45f-f6e8ba3666c3 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:47.778560 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803cfc4-6abe-4800-a45f-f6e8ba3666c3 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:47.778836 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7803cfc4-6abe-4800-a45f-f6e8ba3666c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:47.797863 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-7803cfc4-6abe-4800-a45f-f6e8ba3666c3 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:47.799081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-7803cfc4-6abe-4800-a45f-f6e8ba3666c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:47.870019 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 361/725] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:47 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:53:47.988201 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9a5e0c-5d35-41b3-86b4-e5a1c2f8d29d None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:48.095550 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9a5e0c-5d35-41b3-86b4-e5a1c2f8d29d None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:48.131159 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d5878-6a80-46a2-9e99-fb994a8b454e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:48.131715 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d5878-6a80-46a2-9e99-fb994a8b454e None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:48.131995 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d5878-6a80-46a2-9e99-fb994a8b454e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:48.132267 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930d5878-6a80-46a2-9e99-fb994a8b454e None admin] PATH_INFO: `/v3/services/image` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:48.132853 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930d5878-6a80-46a2-9e99-fb994a8b454e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:48.134225 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930d5878-6a80-46a2-9e99-fb994a8b454e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:48.144033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-930d5878-6a80-46a2-9e99-fb994a8b454e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 23 19:53:48.144033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:48.144033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 19:53:48.144033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 19:53:48.144033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 19:53:48.144033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 19:53:48.144033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 19:53:48.144033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:48.144033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 23 19:53:48.144033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:48.144033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 19:53:48.144033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:48.144033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:48.144033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:48.144033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:48.144033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:48.144033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:48.145023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:48.145023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:48.145023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:48.145023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:48.145023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:48.145023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:48.145023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 19:53:48.145023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 19:53:48.145023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 19:53:48.145023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 19:53:48.145023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:48.145023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:48.145023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:48.145023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:48.145023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:48.145023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:48.145874 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:48.145874 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:48.145874 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:48.145874 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:48.145874 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:48.145874 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:48.145874 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:48.145874 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:48.145874 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:48.145874 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:48.145874 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 19:53:48.145874 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:48.145874 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 23 19:53:48.145874 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:48.147124 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7803cfc4-6abe-4800-a45f-f6e8ba3666c3 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:48.150027 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 365/726] 10.222.0.13 () {56 vars in 1105 bytes} [Sun Oct 23 19:53:47 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:48.151233 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d5878-6a80-46a2-9e99-fb994a8b454e None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:48.195702 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:48.220867 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:48.262451 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930d5878-6a80-46a2-9e99-fb994a8b454e None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:48.276683 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7803cfc4-6abe-4800-a45f-f6e8ba3666c3 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:48.305954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d977293-6bea-4ab3-b8f1-e3f2097a6303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:48.307059 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d977293-6bea-4ab3-b8f1-e3f2097a6303 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:48.307365 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d977293-6bea-4ab3-b8f1-e3f2097a6303 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:48.307645 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d977293-6bea-4ab3-b8f1-e3f2097a6303 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:48.308288 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d977293-6bea-4ab3-b8f1-e3f2097a6303 None admin] RBAC: Adding query filter params (name=image) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:48.308723 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d977293-6bea-4ab3-b8f1-e3f2097a6303 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:48.310397 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d977293-6bea-4ab3-b8f1-e3f2097a6303 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:48.321716 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 366/727] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:48.323346 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d977293-6bea-4ab3-b8f1-e3f2097a6303 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:48.325042 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e3e1e-d5b7-4651-a061-5fda1b90f922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:48.326223 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e3e1e-d5b7-4651-a061-5fda1b90f922 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:48.326678 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e3e1e-d5b7-4651-a061-5fda1b90f922 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:48.328534 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e3e1e-d5b7-4651-a061-5fda1b90f922 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:48.329292 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e3e1e-d5b7-4651-a061-5fda1b90f922 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:48.330864 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e3e1e-d5b7-4651-a061-5fda1b90f922 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:48.343295 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-ca9e3e1e-d5b7-4651-a061-5fda1b90f922 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 19:53:48.343295 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:48.343295 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 19:53:48.343295 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 19:53:48.343295 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 19:53:48.343295 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 19:53:48.343295 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 19:53:48.343295 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:48.343295 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 19:53:48.343295 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:48.343295 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 19:53:48.343295 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:48.343295 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:48.343295 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:48.343295 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:48.343295 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:48.343295 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:48.344548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:48.344548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:48.344548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:48.344548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:48.344548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:48.344548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:48.344548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 19:53:48.344548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 19:53:48.344548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 19:53:48.344548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 19:53:48.344548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:48.344548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:48.344548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:48.344548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:48.344548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:48.344548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:48.345604 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:48.345604 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:48.345604 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:48.345604 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:48.345604 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:48.345604 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:48.345604 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:48.345604 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:48.345604 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:48.345604 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:48.345604 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 19:53:48.345604 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:48.345604 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 19:53:48.345604 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:48.347509 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 362/728] 10.222.0.13 () {56 vars in 1126 bytes} [Sun Oct 23 19:53:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:53:48.353326 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e3e1e-d5b7-4651-a061-5fda1b90f922 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:48.453723 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d977293-6bea-4ab3-b8f1-e3f2097a6303 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:48.456199 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:48.465451 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e3e1e-d5b7-4651-a061-5fda1b90f922 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:48.491034 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:48.496909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c0300d-f0ad-4d9e-a394-ce8dda587b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:48.497748 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0300d-f0ad-4d9e-a394-ce8dda587b32 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:48.498169 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0300d-f0ad-4d9e-a394-ce8dda587b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:48.498614 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c0300d-f0ad-4d9e-a394-ce8dda587b32 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:48.500039 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c0300d-f0ad-4d9e-a394-ce8dda587b32 None admin] RBAC: Adding query filter params (type=image) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:48.500488 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c0300d-f0ad-4d9e-a394-ce8dda587b32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:48.502157 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c0300d-f0ad-4d9e-a394-ce8dda587b32 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:48.505050 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0773a163-516b-4740-9120-26b29e52afb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:48.505781 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773a163-516b-4740-9120-26b29e52afb2 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:48.506172 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773a163-516b-4740-9120-26b29e52afb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:48.506566 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773a163-516b-4740-9120-26b29e52afb2 None admin] PATH_INFO: `/v3/services` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:48.507305 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0773a163-516b-4740-9120-26b29e52afb2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:48.507714 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0773a163-516b-4740-9120-26b29e52afb2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:48.509350 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0773a163-516b-4740-9120-26b29e52afb2 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:48.514493 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 367/729] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:48 2022] GET /identity/v3/services?type=image => generated 122 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:48.522456 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 363/730] 10.222.0.13 () {56 vars in 1122 bytes} [Sun Oct 23 19:53:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:48.528352 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c0300d-f0ad-4d9e-a394-ce8dda587b32 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:48.646825 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c0300d-f0ad-4d9e-a394-ce8dda587b32 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:48.693929 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96fcfcf-2927-405b-a5c2-25c6321abbfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:48.694967 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fcfcf-2927-405b-a5c2-25c6321abbfb None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:48.695458 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fcfcf-2927-405b-a5c2-25c6321abbfb None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:48.695967 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96fcfcf-2927-405b-a5c2-25c6321abbfb None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:48.696821 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96fcfcf-2927-405b-a5c2-25c6321abbfb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:48.697331 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96fcfcf-2927-405b-a5c2-25c6321abbfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:48.698956 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96fcfcf-2927-405b-a5c2-25c6321abbfb None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:48.705387 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 368/731] 10.222.0.13 () {56 vars in 1122 bytes} [Sun Oct 23 19:53:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:48.710665 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0773a163-516b-4740-9120-26b29e52afb2 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:48.777867 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c80552-6fe8-422b-a6ae-de639b24227c None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:48.778542 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c80552-6fe8-422b-a6ae-de639b24227c None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:48.779037 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c80552-6fe8-422b-a6ae-de639b24227c None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:48.780979 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 369/732] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 19:53:48.787444 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387e333-fcdd-4d7b-ab19-07b58d66b64b None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:48.787745 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387e333-fcdd-4d7b-ab19-07b58d66b64b None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:48.788023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a387e333-fcdd-4d7b-ab19-07b58d66b64b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:48.810793 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-a387e333-fcdd-4d7b-ab19-07b58d66b64b None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:48.812215 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a387e333-fcdd-4d7b-ab19-07b58d66b64b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:48.831153 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0773a163-516b-4740-9120-26b29e52afb2 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:48.846841 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:48.875071 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a72cd6-740a-48f0-b768-1c304d2b803d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:48.875852 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a72cd6-740a-48f0-b768-1c304d2b803d None admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:48.876135 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a72cd6-740a-48f0-b768-1c304d2b803d None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:48.876772 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a72cd6-740a-48f0-b768-1c304d2b803d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:48.877606 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a72cd6-740a-48f0-b768-1c304d2b803d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:48.879277 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a72cd6-740a-48f0-b768-1c304d2b803d None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:48.902910 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 370/733] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:48 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:53:48.905185 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 364/734] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 19:53:48 2022] POST /identity/v3/endpoints => generated 349 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 19:53:49.174435 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387e333-fcdd-4d7b-ab19-07b58d66b64b None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:49.278797 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a387e333-fcdd-4d7b-ab19-07b58d66b64b None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:49.323419 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259fa9a1-c749-47f7-99cd-419b9c64b550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:49.324040 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fa9a1-c749-47f7-99cd-419b9c64b550 None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:49.324319 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fa9a1-c749-47f7-99cd-419b9c64b550 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:49.324593 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259fa9a1-c749-47f7-99cd-419b9c64b550 None admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:49.325251 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259fa9a1-c749-47f7-99cd-419b9c64b550 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:49.326795 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259fa9a1-c749-47f7-99cd-419b9c64b550 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:49.329487 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: WARNING keystone.server.flask.common [None req-259fa9a1-c749-47f7-99cd-419b9c64b550 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 23 19:53:49.348548 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.sql.core [None req-259fa9a1-c749-47f7-99cd-419b9c64b550 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 23 19:53:49.348548 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 23 19:53:49.348548 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [parameters: {'id': 'd6b69c6df982428d918a494751315c1c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 23 19:53:49.348548 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81411) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 23 19:53:49.357466 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-259fa9a1-c749-47f7-99cd-419b9c64b550 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d6b69c6df982428d918a494751315c1c.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d6b69c6df982428d918a494751315c1c. Oct 23 19:53:49.357466 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:49.357466 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 23 19:53:49.357466 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 23 19:53:49.357466 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 23 19:53:49.357466 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 23 19:53:49.357466 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 23 19:53:49.357466 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application result = self._query(query) Oct 23 19:53:49.357466 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 23 19:53:49.357466 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application conn.query(q) Oct 23 19:53:49.357466 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 23 19:53:49.357466 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 23 19:53:49.357466 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 23 19:53:49.358315 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application result.read() Oct 23 19:53:49.358315 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 23 19:53:49.358315 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 23 19:53:49.358315 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 23 19:53:49.358315 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 23 19:53:49.358315 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 23 19:53:49.358315 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 23 19:53:49.358315 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 23 19:53:49.358315 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 23 19:53:49.358315 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 23 19:53:49.358315 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:49.358315 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 23 19:53:49.358315 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:49.358315 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:49.358315 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 23 19:53:49.358315 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 23 19:53:49.358315 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 23 19:53:49.358315 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 23 19:53:49.359322 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 23 19:53:49.359322 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application next(self.gen) Oct 23 19:53:49.359322 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 23 19:53:49.359322 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application yield resource Oct 23 19:53:49.359322 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 23 19:53:49.359322 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application next(self.gen) Oct 23 19:53:49.359322 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 23 19:53:49.359322 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self.session.rollback() Oct 23 19:53:49.359322 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 23 19:53:49.359322 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self.force_reraise() Oct 23 19:53:49.359322 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 23 19:53:49.359322 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise self.value Oct 23 19:53:49.359322 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 23 19:53:49.359322 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 23 19:53:49.359322 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 23 19:53:49.359322 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application session.commit() Oct 23 19:53:49.359322 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 23 19:53:49.360243 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 23 19:53:49.360243 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 23 19:53:49.360243 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._prepare_impl() Oct 23 19:53:49.360243 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 23 19:53:49.360243 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self.session.flush() Oct 23 19:53:49.360243 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 23 19:53:49.360243 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._flush(objects) Oct 23 19:53:49.360243 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 23 19:53:49.360243 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 23 19:53:49.360243 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 23 19:53:49.360243 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application compat.raise_( Oct 23 19:53:49.360243 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 23 19:53:49.360243 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception Oct 23 19:53:49.360243 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 23 19:53:49.360243 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application flush_context.execute() Oct 23 19:53:49.360243 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 23 19:53:49.360243 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rec.execute(self) Oct 23 19:53:49.361139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 23 19:53:49.361139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 23 19:53:49.361139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 23 19:53:49.361139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 23 19:53:49.361139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 23 19:53:49.361139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 23 19:53:49.361139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 23 19:53:49.361139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 23 19:53:49.361139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 23 19:53:49.361139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 23 19:53:49.361139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 23 19:53:49.361139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 23 19:53:49.361139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 23 19:53:49.361139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 23 19:53:49.361139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 23 19:53:49.361954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 23 19:53:49.361954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 23 19:53:49.361954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception Oct 23 19:53:49.361954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 23 19:53:49.361954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 23 19:53:49.361954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 23 19:53:49.361954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 23 19:53:49.361954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 23 19:53:49.361954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application result = self._query(query) Oct 23 19:53:49.361954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 23 19:53:49.361954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application conn.query(q) Oct 23 19:53:49.361954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 23 19:53:49.361954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 23 19:53:49.361954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 23 19:53:49.361954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application result.read() Oct 23 19:53:49.361954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 23 19:53:49.361954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 23 19:53:49.364706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 23 19:53:49.364706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 23 19:53:49.364706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 23 19:53:49.364706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 23 19:53:49.364706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 23 19:53:49.364706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 23 19:53:49.364706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 23 19:53:49.364706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 23 19:53:49.364706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [parameters: {'id': 'd6b69c6df982428d918a494751315c1c', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 23 19:53:49.364706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 23 19:53:49.364706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:49.364706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 19:53:49.364706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:49.364706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:49.364706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 23 19:53:49.366577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 23 19:53:49.366577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 23 19:53:49.366577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 23 19:53:49.366577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 23 19:53:49.366577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:49.366577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 19:53:49.366577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:49.366577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:49.366577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:49.366577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:49.366577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:49.366577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:49.366577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:49.366577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:49.366577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:49.366577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:49.366577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:49.375512 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:49.375512 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 23 19:53:49.375512 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 23 19:53:49.375512 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:49.375512 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:49.375512 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 23 19:53:49.375512 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 23 19:53:49.375512 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id d6b69c6df982428d918a494751315c1c. Oct 23 19:53:49.375512 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:49.375512 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 371/735] 10.222.0.13 () {60 vars in 1140 bytes} [Sun Oct 23 19:53:49 2022] POST /identity/v3/projects => generated 262 bytes in 185 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:49.376859 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a72cd6-740a-48f0-b768-1c304d2b803d None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:49.427323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:49.496459 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a72cd6-740a-48f0-b768-1c304d2b803d None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:49.538326 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4832e76e-f80a-41c1-bb88-3cdf63ef23cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:49.539316 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832e76e-f80a-41c1-bb88-3cdf63ef23cf None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:49.539796 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832e76e-f80a-41c1-bb88-3cdf63ef23cf None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:49.540310 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832e76e-f80a-41c1-bb88-3cdf63ef23cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:49.548189 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4832e76e-f80a-41c1-bb88-3cdf63ef23cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:49.549841 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4832e76e-f80a-41c1-bb88-3cdf63ef23cf None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:49.561698 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-4832e76e-f80a-41c1-bb88-3cdf63ef23cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:49.561698 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:49.561698 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:49.561698 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:49.561698 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:49.561698 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:49.561698 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:49.561698 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:49.561698 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:49.561698 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:49.561698 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:49.561698 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:49.561698 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 19:53:49.561698 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 19:53:49.561698 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 19:53:49.562618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 19:53:49.562618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:49.562618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:49.562618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:49.562618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:49.562618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:49.562618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:49.562618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:49.562618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:49.562618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:49.562618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:49.562618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:49.562618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:49.562618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:49.562618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:49.562618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:49.562618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:49.563535 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 19:53:49.563535 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 19:53:49.563535 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 19:53:49.563535 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 19:53:49.563535 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 19:53:49.563535 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 19:53:49.563535 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:49.563535 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:49.566120 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 365/736] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:53:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:49.572254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259fa9a1-c749-47f7-99cd-419b9c64b550 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:49.590232 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:49.598132 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e3c50-575f-41f7-8066-b1edb2239540 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:49.598448 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e3c50-575f-41f7-8066-b1edb2239540 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:49.598722 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412e3c50-575f-41f7-8066-b1edb2239540 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:49.600327 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 366/737] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 19:53:49.606189 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfed49c-93ab-40db-bf9f-3beec2dc57ba None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:49.606502 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfed49c-93ab-40db-bf9f-3beec2dc57ba None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:49.606774 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfed49c-93ab-40db-bf9f-3beec2dc57ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:49.625558 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-2bfed49c-93ab-40db-bf9f-3beec2dc57ba None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:49.626771 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-2bfed49c-93ab-40db-bf9f-3beec2dc57ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:49.691086 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259fa9a1-c749-47f7-99cd-419b9c64b550 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:49.718289 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 367/738] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:49 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 19:53:49.737451 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc0f07-c0a7-4486-afa8-2f40bc100def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:49.738481 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc0f07-c0a7-4486-afa8-2f40bc100def None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:49.738988 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc0f07-c0a7-4486-afa8-2f40bc100def None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:49.739511 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefc0f07-c0a7-4486-afa8-2f40bc100def None admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:49.740372 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefc0f07-c0a7-4486-afa8-2f40bc100def None admin] RBAC: Adding query filter params (name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:49.740882 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefc0f07-c0a7-4486-afa8-2f40bc100def None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:49.742794 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefc0f07-c0a7-4486-afa8-2f40bc100def None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:49.764282 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 372/739] 10.222.0.13 () {56 vars in 1112 bytes} [Sun Oct 23 19:53:49 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:49.985605 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfed49c-93ab-40db-bf9f-3beec2dc57ba None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:50.027721 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:50.100980 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfed49c-93ab-40db-bf9f-3beec2dc57ba None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:50.136845 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699c611-5f51-47d1-b57c-983b952c3657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:50.137483 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699c611-5f51-47d1-b57c-983b952c3657 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:50.137763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699c611-5f51-47d1-b57c-983b952c3657 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:50.138038 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2699c611-5f51-47d1-b57c-983b952c3657 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:50.138654 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2699c611-5f51-47d1-b57c-983b952c3657 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:50.140084 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2699c611-5f51-47d1-b57c-983b952c3657 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:50.149239 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-2699c611-5f51-47d1-b57c-983b952c3657 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 23 19:53:50.149239 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:50.149239 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 19:53:50.149239 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 19:53:50.149239 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 19:53:50.149239 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 19:53:50.149239 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 19:53:50.149239 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:50.149239 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 23 19:53:50.149239 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:50.149239 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 19:53:50.149239 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:50.149239 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:50.149239 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:50.149239 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:50.149239 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:50.149239 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:50.150215 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:50.150215 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:50.150215 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:50.150215 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:50.150215 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:50.150215 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:50.150215 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 19:53:50.150215 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 19:53:50.150215 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 19:53:50.150215 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 19:53:50.150215 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:50.150215 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:50.150215 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:50.150215 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:50.150215 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:50.150215 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:50.151081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:50.151081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:50.151081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:50.151081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:50.151081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:50.151081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:50.151081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:50.151081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:50.151081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:50.151081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:50.151081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 19:53:50.151081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:50.151081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 23 19:53:50.151081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:50.151081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 368/740] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:53:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:50.158860 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc0f07-c0a7-4486-afa8-2f40bc100def None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:50.240692 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:50.268149 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefc0f07-c0a7-4486-afa8-2f40bc100def None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:50.303310 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17cedc-047e-4a48-b6e3-818e428b6e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:50.303926 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17cedc-047e-4a48-b6e3-818e428b6e06 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:50.304210 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17cedc-047e-4a48-b6e3-818e428b6e06 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:50.304482 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d17cedc-047e-4a48-b6e3-818e428b6e06 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:50.305108 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d17cedc-047e-4a48-b6e3-818e428b6e06 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:50.305412 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d17cedc-047e-4a48-b6e3-818e428b6e06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:50.306799 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d17cedc-047e-4a48-b6e3-818e428b6e06 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:50.312233 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 373/741] 10.222.0.13 () {56 vars in 1112 bytes} [Sun Oct 23 19:53:50 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:50.319440 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699c611-5f51-47d1-b57c-983b952c3657 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:50.424552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:50.428635 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2699c611-5f51-47d1-b57c-983b952c3657 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:50.464144 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd77237-1202-46c9-a2f3-c13009bb431f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:50.464787 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd77237-1202-46c9-a2f3-c13009bb431f None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:50.465074 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd77237-1202-46c9-a2f3-c13009bb431f None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:50.465346 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd77237-1202-46c9-a2f3-c13009bb431f None admin] PATH_INFO: `/v3/services` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:50.466027 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd77237-1202-46c9-a2f3-c13009bb431f None admin] RBAC: Adding query filter params (type=compute) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:50.466354 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd77237-1202-46c9-a2f3-c13009bb431f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:50.467749 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd77237-1202-46c9-a2f3-c13009bb431f None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:50.473048 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 369/742] 10.222.0.13 () {56 vars in 1112 bytes} [Sun Oct 23 19:53:50 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:50.479402 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17cedc-047e-4a48-b6e3-818e428b6e06 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:50.583761 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d17cedc-047e-4a48-b6e3-818e428b6e06 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:50.619752 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e57bee-55ff-4291-bf67-66af57bbe507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:50.620740 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e57bee-55ff-4291-bf67-66af57bbe507 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:50.621236 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e57bee-55ff-4291-bf67-66af57bbe507 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:50.621705 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e57bee-55ff-4291-bf67-66af57bbe507 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:50.622697 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e57bee-55ff-4291-bf67-66af57bbe507 None admin] RBAC: Adding query filter params (interface=public, service_id=971f66b11dbc4dd292a8a0c5978f17b2, region_id=RegionOne) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:50.623215 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e57bee-55ff-4291-bf67-66af57bbe507 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:50.624830 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e57bee-55ff-4291-bf67-66af57bbe507 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:50.630757 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 374/743] 10.222.0.13 () {56 vars in 1251 bytes} [Sun Oct 23 19:53:50 2022] GET /identity/v3/endpoints?service_id=971f66b11dbc4dd292a8a0c5978f17b2&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:50.636664 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd77237-1202-46c9-a2f3-c13009bb431f None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:50.749911 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd77237-1202-46c9-a2f3-c13009bb431f None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:50.785137 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f40d527-ce64-4b85-800b-547735f3d23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:50.785758 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40d527-ce64-4b85-800b-547735f3d23b None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:50.786037 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40d527-ce64-4b85-800b-547735f3d23b None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:50.786335 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f40d527-ce64-4b85-800b-547735f3d23b None admin] PATH_INFO: `/v3/services` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:50.786949 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f40d527-ce64-4b85-800b-547735f3d23b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:50.788304 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f40d527-ce64-4b85-800b-547735f3d23b None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:50.795007 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 370/744] 10.222.0.13 () {56 vars in 1087 bytes} [Sun Oct 23 19:53:50 2022] GET /identity/v3/services => generated 1727 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 19:53:50.806486 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30214ca1-3b63-45ea-af1e-4173df868625 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:50.807003 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30214ca1-3b63-45ea-af1e-4173df868625 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:50.807104 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30214ca1-3b63-45ea-af1e-4173df868625 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:50.809957 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 375/745] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:50.815601 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d20444-2cf3-4427-a62d-27abb39be918 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:50.815954 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d20444-2cf3-4427-a62d-27abb39be918 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:50.816235 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d20444-2cf3-4427-a62d-27abb39be918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:50.837146 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-a4d20444-2cf3-4427-a62d-27abb39be918 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:50.838327 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-a4d20444-2cf3-4427-a62d-27abb39be918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:50.862269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:50.914727 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 371/746] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:50 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:53:51.187851 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30214ca1-3b63-45ea-af1e-4173df868625 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:51.191064 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:51.201254 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed44567-a778-4954-9063-773d4c0a979a None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:51.201560 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed44567-a778-4954-9063-773d4c0a979a None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:51.201823 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed44567-a778-4954-9063-773d4c0a979a None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:51.203531 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 372/747] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 19:53:51.208585 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647dc02-8246-44cf-b57c-397ab90ba3b9 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:51.208874 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647dc02-8246-44cf-b57c-397ab90ba3b9 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:51.209149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9647dc02-8246-44cf-b57c-397ab90ba3b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:51.226890 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-9647dc02-8246-44cf-b57c-397ab90ba3b9 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:51.228089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-9647dc02-8246-44cf-b57c-397ab90ba3b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:51.302369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 373/748] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:51 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 19:53:51.311844 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30214ca1-3b63-45ea-af1e-4173df868625 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:51.347134 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4792fe-5925-4470-baf5-3a6f52e9360b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:51.347742 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4792fe-5925-4470-baf5-3a6f52e9360b None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:51.348021 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4792fe-5925-4470-baf5-3a6f52e9360b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:51.348295 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4792fe-5925-4470-baf5-3a6f52e9360b None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:51.348954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4792fe-5925-4470-baf5-3a6f52e9360b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:51.350356 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4792fe-5925-4470-baf5-3a6f52e9360b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:51.370648 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 376/749] 10.222.0.13 () {60 vars in 1141 bytes} [Sun Oct 23 19:53:51 2022] POST /identity/v3/services => generated 242 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:53:51.574388 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9647dc02-8246-44cf-b57c-397ab90ba3b9 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:51.664195 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:51.679772 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9647dc02-8246-44cf-b57c-397ab90ba3b9 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:51.721973 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec1585-f9cc-40e6-8557-595dcab3dfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:51.722647 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec1585-f9cc-40e6-8557-595dcab3dfca None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:51.722926 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec1585-f9cc-40e6-8557-595dcab3dfca None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:51.723198 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec1585-f9cc-40e6-8557-595dcab3dfca None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:51.723786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec1585-f9cc-40e6-8557-595dcab3dfca None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:51.725144 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec1585-f9cc-40e6-8557-595dcab3dfca None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:51.734458 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-57ec1585-f9cc-40e6-8557-595dcab3dfca None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 19:53:51.734458 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:51.734458 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 19:53:51.734458 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 19:53:51.734458 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 19:53:51.734458 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 19:53:51.734458 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 19:53:51.734458 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:51.734458 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 19:53:51.734458 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:51.734458 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 19:53:51.734458 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:51.734458 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:51.734458 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:51.734458 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:51.734458 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:51.734458 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:51.735476 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:51.735476 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:51.735476 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:51.735476 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:51.735476 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:51.735476 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:51.735476 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 19:53:51.735476 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 19:53:51.735476 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 19:53:51.735476 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 19:53:51.735476 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:51.735476 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:51.735476 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:51.735476 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:51.735476 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:51.735476 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:51.736323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:51.736323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:51.736323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:51.736323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:51.736323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:51.736323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:51.736323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:51.736323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:51.736323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:51.736323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:51.736323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 19:53:51.736323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:51.736323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 19:53:51.736323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:51.736323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 374/750] 10.222.0.13 () {56 vars in 1126 bytes} [Sun Oct 23 19:53:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:51.742153 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4792fe-5925-4470-baf5-3a6f52e9360b None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:51.806359 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:51.852257 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4792fe-5925-4470-baf5-3a6f52e9360b None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:51.887007 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09616f38-1987-4f23-b893-1e9ad5b8798a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:51.887617 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09616f38-1987-4f23-b893-1e9ad5b8798a None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:51.887896 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09616f38-1987-4f23-b893-1e9ad5b8798a None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:51.888172 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09616f38-1987-4f23-b893-1e9ad5b8798a None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:51.888803 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09616f38-1987-4f23-b893-1e9ad5b8798a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:51.889104 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09616f38-1987-4f23-b893-1e9ad5b8798a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:51.891019 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09616f38-1987-4f23-b893-1e9ad5b8798a None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:51.896288 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 377/751] 10.222.0.13 () {56 vars in 1122 bytes} [Sun Oct 23 19:53:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:51.907297 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec1585-f9cc-40e6-8557-595dcab3dfca None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:52.010902 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec1585-f9cc-40e6-8557-595dcab3dfca None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:52.047122 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b280b069-cf4a-4af5-aa1e-b7fb6338a4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:52.047740 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280b069-cf4a-4af5-aa1e-b7fb6338a4c5 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:52.048021 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280b069-cf4a-4af5-aa1e-b7fb6338a4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:52.048294 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b280b069-cf4a-4af5-aa1e-b7fb6338a4c5 None admin] PATH_INFO: `/v3/services` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:52.048930 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280b069-cf4a-4af5-aa1e-b7fb6338a4c5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:52.049236 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280b069-cf4a-4af5-aa1e-b7fb6338a4c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:52.050621 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b280b069-cf4a-4af5-aa1e-b7fb6338a4c5 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:52.056101 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 375/752] 10.222.0.13 () {56 vars in 1122 bytes} [Sun Oct 23 19:53:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:52.062173 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09616f38-1987-4f23-b893-1e9ad5b8798a None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:52.176232 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09616f38-1987-4f23-b893-1e9ad5b8798a None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:52.206604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:52.215527 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5347898d-2b8b-43e3-9690-ecc5d18169d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:52.216412 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347898d-2b8b-43e3-9690-ecc5d18169d9 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:52.216922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347898d-2b8b-43e3-9690-ecc5d18169d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:52.217408 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5347898d-2b8b-43e3-9690-ecc5d18169d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:52.218866 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5347898d-2b8b-43e3-9690-ecc5d18169d9 None admin] RBAC: Adding query filter params (interface=admin, service_id=49c001dad7a5485684b3fd87e3ffc6f7, region_id=RegionOne) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:52.219537 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5347898d-2b8b-43e3-9690-ecc5d18169d9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:52.220983 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e030e9-554a-49ad-b7fe-b176e39f8d5e None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:52.221277 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e030e9-554a-49ad-b7fe-b176e39f8d5e None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:52.221555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e030e9-554a-49ad-b7fe-b176e39f8d5e None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:52.221777 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5347898d-2b8b-43e3-9690-ecc5d18169d9 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:52.223152 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 376/753] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:52.228049 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc59cb-9da0-4579-a3a9-5c12f8703ba2 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:52.228334 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc59cb-9da0-4579-a3a9-5c12f8703ba2 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:52.228608 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bc59cb-9da0-4579-a3a9-5c12f8703ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:52.233563 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 378/754] 10.222.0.13 () {56 vars in 1249 bytes} [Sun Oct 23 19:53:52 2022] GET /identity/v3/endpoints?service_id=49c001dad7a5485684b3fd87e3ffc6f7&interface=admin®ion_id=RegionOne => generated 193 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:52.235015 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5347898d-2b8b-43e3-9690-ecc5d18169d9 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:52.247605 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-37bc59cb-9da0-4579-a3a9-5c12f8703ba2 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:52.248799 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-37bc59cb-9da0-4579-a3a9-5c12f8703ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:52.252444 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:52.330322 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 377/755] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:52 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 19:53:52.348725 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5347898d-2b8b-43e3-9690-ecc5d18169d9 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:52.382630 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca7256-a674-445f-a5da-9f9bb5842e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:52.383418 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca7256-a674-445f-a5da-9f9bb5842e34 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:52.383877 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca7256-a674-445f-a5da-9f9bb5842e34 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:52.384297 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ca7256-a674-445f-a5da-9f9bb5842e34 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:52.385038 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca7256-a674-445f-a5da-9f9bb5842e34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:52.386557 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ca7256-a674-445f-a5da-9f9bb5842e34 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:52.393608 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 379/756] 10.222.0.13 () {56 vars in 1087 bytes} [Sun Oct 23 19:53:52 2022] GET /identity/v3/services => generated 1957 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 19:53:52.598795 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc59cb-9da0-4579-a3a9-5c12f8703ba2 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:52.701329 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bc59cb-9da0-4579-a3a9-5c12f8703ba2 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:52.737034 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad9be0-dc9d-4f54-b4a6-2c54df6e756b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:52.737722 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad9be0-dc9d-4f54-b4a6-2c54df6e756b None admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:52.738042 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad9be0-dc9d-4f54-b4a6-2c54df6e756b None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:52.738346 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad9be0-dc9d-4f54-b4a6-2c54df6e756b None admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:52.738999 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ad9be0-dc9d-4f54-b4a6-2c54df6e756b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:52.740424 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ad9be0-dc9d-4f54-b4a6-2c54df6e756b None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:52.742994 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: WARNING keystone.server.flask.common [None req-41ad9be0-dc9d-4f54-b4a6-2c54df6e756b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 23 19:53:52.756381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.sql.core [None req-41ad9be0-dc9d-4f54-b4a6-2c54df6e756b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 23 19:53:52.756381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 23 19:53:52.756381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [parameters: {'user_id': '1f09ea98dc274988be739d1e21800e2a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 23 19:53:52.756381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81412) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 23 19:53:52.763954 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-41ad9be0-dc9d-4f54-b4a6-2c54df6e756b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 23 19:53:52.763954 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:52.763954 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 23 19:53:52.763954 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 23 19:53:52.763954 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 23 19:53:52.763954 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 23 19:53:52.763954 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 23 19:53:52.763954 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application result = self._query(query) Oct 23 19:53:52.763954 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 23 19:53:52.763954 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application conn.query(q) Oct 23 19:53:52.763954 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 23 19:53:52.763954 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 23 19:53:52.763954 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 23 19:53:52.763954 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application result.read() Oct 23 19:53:52.763954 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 23 19:53:52.765373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 23 19:53:52.765373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 23 19:53:52.765373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 23 19:53:52.765373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 23 19:53:52.765373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 23 19:53:52.765373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 23 19:53:52.765373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 23 19:53:52.765373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 23 19:53:52.765373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:52.765373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 23 19:53:52.765373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:52.765373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:52.765373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 23 19:53:52.765373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 23 19:53:52.765373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 23 19:53:52.765373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 23 19:53:52.765373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 23 19:53:52.765373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application next(self.gen) Oct 23 19:53:52.766747 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 23 19:53:52.766747 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application yield resource Oct 23 19:53:52.766747 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 23 19:53:52.766747 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application next(self.gen) Oct 23 19:53:52.766747 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 23 19:53:52.766747 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self.session.rollback() Oct 23 19:53:52.766747 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 23 19:53:52.766747 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self.force_reraise() Oct 23 19:53:52.766747 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 23 19:53:52.766747 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise self.value Oct 23 19:53:52.766747 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 23 19:53:52.766747 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 23 19:53:52.766747 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 23 19:53:52.766747 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application session.commit() Oct 23 19:53:52.766747 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 23 19:53:52.766747 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 23 19:53:52.766747 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 23 19:53:52.768912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self._prepare_impl() Oct 23 19:53:52.768912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 23 19:53:52.768912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self.session.flush() Oct 23 19:53:52.768912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 23 19:53:52.768912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self._flush(objects) Oct 23 19:53:52.768912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 23 19:53:52.768912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 23 19:53:52.768912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 23 19:53:52.768912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application compat.raise_( Oct 23 19:53:52.768912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 23 19:53:52.768912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception Oct 23 19:53:52.768912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 23 19:53:52.768912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application flush_context.execute() Oct 23 19:53:52.768912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 23 19:53:52.768912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rec.execute(self) Oct 23 19:53:52.768912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 23 19:53:52.768912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 23 19:53:52.774555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 23 19:53:52.774555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 23 19:53:52.774555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 23 19:53:52.774555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 23 19:53:52.774555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 23 19:53:52.774555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 23 19:53:52.774555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 23 19:53:52.774555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 23 19:53:52.774555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 23 19:53:52.774555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 23 19:53:52.774555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 23 19:53:52.774555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 23 19:53:52.774555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 23 19:53:52.774555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 23 19:53:52.774555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 23 19:53:52.774555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception Oct 23 19:53:52.775585 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 23 19:53:52.775585 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 23 19:53:52.775585 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 23 19:53:52.775585 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 23 19:53:52.775585 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 23 19:53:52.775585 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application result = self._query(query) Oct 23 19:53:52.775585 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 23 19:53:52.775585 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application conn.query(q) Oct 23 19:53:52.775585 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 23 19:53:52.775585 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 23 19:53:52.775585 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 23 19:53:52.775585 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application result.read() Oct 23 19:53:52.775585 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 23 19:53:52.775585 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 23 19:53:52.775585 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 23 19:53:52.775585 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 23 19:53:52.776447 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 23 19:53:52.776447 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 23 19:53:52.776447 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 23 19:53:52.776447 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 23 19:53:52.776447 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 23 19:53:52.776447 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 23 19:53:52.776447 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [parameters: {'user_id': '1f09ea98dc274988be739d1e21800e2a', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 23 19:53:52.776447 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 23 19:53:52.776447 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:52.776447 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 19:53:52.776447 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:52.776447 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:52.776447 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:52.776447 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:52.776447 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:52.777266 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:52.777266 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:52.777266 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:52.777266 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:52.777266 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:52.777266 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:52.777266 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:52.777266 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 23 19:53:52.777266 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 23 19:53:52.777266 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:52.777266 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:52.777266 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 19:53:52.777266 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:52.777266 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 19:53:52.777266 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:52.777266 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 23 19:53:52.777266 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 23 19:53:52.778285 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 23 19:53:52.778285 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 23 19:53:52.778285 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 23 19:53:52.778285 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 23 19:53:52.778285 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 23 19:53:52.778285 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:52.778285 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 378/757] 10.222.0.13 () {60 vars in 1131 bytes} [Sun Oct 23 19:53:52 2022] POST /identity/v3/users => generated 160 bytes in 168 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:52.778699 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca7256-a674-445f-a5da-9f9bb5842e34 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:52.837662 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:52.887778 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ca7256-a674-445f-a5da-9f9bb5842e34 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:52.922764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1a7511-5eb3-4327-98b5-a1f91721e700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:52.923687 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a7511-5eb3-4327-98b5-a1f91721e700 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:52.924111 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a7511-5eb3-4327-98b5-a1f91721e700 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:52.924559 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1a7511-5eb3-4327-98b5-a1f91721e700 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:52.932658 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1a7511-5eb3-4327-98b5-a1f91721e700 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:52.934327 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1a7511-5eb3-4327-98b5-a1f91721e700 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:52.946704 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-de1a7511-5eb3-4327-98b5-a1f91721e700 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 23 19:53:52.946704 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:52.946704 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:52.946704 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:52.946704 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:52.946704 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:52.946704 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:52.946704 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:52.946704 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:52.946704 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:52.946704 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:52.946704 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:52.946704 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 19:53:52.946704 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 19:53:52.946704 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 19:53:52.946704 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 19:53:52.947889 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:52.947889 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:52.947889 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 19:53:52.947889 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:52.947889 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 19:53:52.947889 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:52.947889 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:52.947889 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:52.947889 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:52.947889 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:52.947889 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:52.947889 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:52.947889 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:52.947889 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:52.947889 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:52.947889 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:52.947889 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:52.947889 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:52.949068 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:52.949068 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:52.949068 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 19:53:52.949068 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 19:53:52.949068 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 19:53:52.949068 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 19:53:52.949068 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 19:53:52.949068 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 19:53:52.949068 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 23 19:53:52.949068 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:52.951236 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 380/758] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:53:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:52.957399 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad9be0-dc9d-4f54-b4a6-2c54df6e756b None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:52.960807 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:53.069972 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad9be0-dc9d-4f54-b4a6-2c54df6e756b None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:53.106351 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02766cf0-ea61-45c1-8d2f-4f9d3da806da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:53.106970 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02766cf0-ea61-45c1-8d2f-4f9d3da806da None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:53.107257 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02766cf0-ea61-45c1-8d2f-4f9d3da806da None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:53.107577 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02766cf0-ea61-45c1-8d2f-4f9d3da806da None admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:53.108323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02766cf0-ea61-45c1-8d2f-4f9d3da806da None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:53.108636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02766cf0-ea61-45c1-8d2f-4f9d3da806da None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:53.110344 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02766cf0-ea61-45c1-8d2f-4f9d3da806da None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:53.150806 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 379/759] 10.222.0.13 () {56 vars in 1101 bytes} [Sun Oct 23 19:53:52 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:53.165842 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244cdfa6-9e62-43b4-bf42-192319ccc305 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:53.166526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244cdfa6-9e62-43b4-bf42-192319ccc305 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:53.167034 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244cdfa6-9e62-43b4-bf42-192319ccc305 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:53.169020 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 381/760] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:53.175309 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800ad01-4285-4e70-9772-e2e738371a19 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:53.175955 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800ad01-4285-4e70-9772-e2e738371a19 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:53.176438 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800ad01-4285-4e70-9772-e2e738371a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:53.197496 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-4800ad01-4285-4e70-9772-e2e738371a19 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:53.199478 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-4800ad01-4285-4e70-9772-e2e738371a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:53.264852 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:53.279662 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 380/761] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:53 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:53:53.551579 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244cdfa6-9e62-43b4-bf42-192319ccc305 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:53.602951 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcf02b-646b-4626-a933-209c415efb29 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:53.603506 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcf02b-646b-4626-a933-209c415efb29 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:53.603506 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fcf02b-646b-4626-a933-209c415efb29 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:53.604856 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 381/762] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:53.610508 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd68743-5283-4b36-bb36-4c808291f74d None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:53.610793 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd68743-5283-4b36-bb36-4c808291f74d None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:53.611063 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd68743-5283-4b36-bb36-4c808291f74d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:53.632084 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-edd68743-5283-4b36-bb36-4c808291f74d None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:53.633948 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-edd68743-5283-4b36-bb36-4c808291f74d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:53.661396 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244cdfa6-9e62-43b4-bf42-192319ccc305 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:53.677151 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:53.700717 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa02754-3a33-4169-b085-4bee4bd899ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:53.701356 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa02754-3a33-4169-b085-4bee4bd899ad None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:53.701640 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa02754-3a33-4169-b085-4bee4bd899ad None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:53.702110 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa02754-3a33-4169-b085-4bee4bd899ad None admin] PATH_INFO: `/v3/services/compute` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:53.702752 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa02754-3a33-4169-b085-4bee4bd899ad None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:53.704195 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa02754-3a33-4169-b085-4bee4bd899ad None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:53.707438 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 382/763] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:53 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 19:53:53.713263 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-1fa02754-3a33-4169-b085-4bee4bd899ad None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 23 19:53:53.713263 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:53.713263 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 19:53:53.713263 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 19:53:53.713263 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 19:53:53.713263 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 19:53:53.713263 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 19:53:53.713263 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:53.713263 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 23 19:53:53.713263 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:53.713263 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 19:53:53.713263 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:53.713263 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:53.713263 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:53.713263 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:53.713263 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:53.713263 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:53.714271 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:53.714271 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:53.714271 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:53.714271 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:53.714271 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:53.714271 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:53.714271 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 19:53:53.714271 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 19:53:53.714271 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 19:53:53.714271 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 19:53:53.714271 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:53.714271 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:53.714271 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:53.714271 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:53.714271 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:53.714271 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:53.719332 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:53.719332 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:53.719332 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:53.719332 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:53.719332 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:53.719332 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:53.719332 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:53.719332 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:53.719332 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:53.719332 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:53.719332 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 19:53:53.719332 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:53.719332 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 23 19:53:53.719332 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:53.719332 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 382/764] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:53:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:53.720679 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd68743-5283-4b36-bb36-4c808291f74d None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:53.822703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd68743-5283-4b36-bb36-4c808291f74d None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:53.842308 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:53.857765 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5a954-311d-4a89-b436-0a9adabe71c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:53.858973 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5a954-311d-4a89-b436-0a9adabe71c0 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:53.859255 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5a954-311d-4a89-b436-0a9adabe71c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:53.859527 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5a954-311d-4a89-b436-0a9adabe71c0 None admin] PATH_INFO: `/v3/services` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:53.860148 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c5a954-311d-4a89-b436-0a9adabe71c0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:53.860449 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c5a954-311d-4a89-b436-0a9adabe71c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:53.861925 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c5a954-311d-4a89-b436-0a9adabe71c0 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:53.867173 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 383/765] 10.222.0.13 () {56 vars in 1112 bytes} [Sun Oct 23 19:53:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:53.872427 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa02754-3a33-4169-b085-4bee4bd899ad None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:53.972079 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa02754-3a33-4169-b085-4bee4bd899ad None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:53.982329 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5a954-311d-4a89-b436-0a9adabe71c0 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:54.012794 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76432a5-1074-461c-831d-ef626c6434e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:54.013411 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76432a5-1074-461c-831d-ef626c6434e1 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:54.013686 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76432a5-1074-461c-831d-ef626c6434e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:54.013956 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76432a5-1074-461c-831d-ef626c6434e1 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:54.015213 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76432a5-1074-461c-831d-ef626c6434e1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:54.015511 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76432a5-1074-461c-831d-ef626c6434e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:54.017325 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76432a5-1074-461c-831d-ef626c6434e1 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:54.022436 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 383/766] 10.222.0.13 () {56 vars in 1112 bytes} [Sun Oct 23 19:53:53 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:54.027943 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76432a5-1074-461c-831d-ef626c6434e1 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:54.081133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:54.094622 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5a954-311d-4a89-b436-0a9adabe71c0 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:54.129884 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cca1ad-8487-4c89-b3c5-67791fbb0496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:54.130551 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cca1ad-8487-4c89-b3c5-67791fbb0496 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:54.130830 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cca1ad-8487-4c89-b3c5-67791fbb0496 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:54.131101 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cca1ad-8487-4c89-b3c5-67791fbb0496 None admin] PATH_INFO: `/v3/services/image` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:54.131686 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cca1ad-8487-4c89-b3c5-67791fbb0496 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:54.133039 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cca1ad-8487-4c89-b3c5-67791fbb0496 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:54.142633 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-93cca1ad-8487-4c89-b3c5-67791fbb0496 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 23 19:53:54.142633 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:54.142633 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 19:53:54.142633 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 19:53:54.142633 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 19:53:54.142633 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 19:53:54.142633 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 19:53:54.142633 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:54.142633 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 23 19:53:54.142633 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:54.142633 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 19:53:54.142633 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:54.142633 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:54.142633 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:54.142633 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:54.142633 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:54.142633 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:54.143632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:54.143632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:54.143632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:54.143632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:54.143632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:54.143632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:54.143632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 19:53:54.143632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 19:53:54.143632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 19:53:54.143632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 19:53:54.143632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:54.143632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:54.143632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:54.143632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:54.143632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:54.143632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:54.144869 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:54.144869 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:54.144869 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:54.144869 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:54.144869 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:54.144869 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:54.144869 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:54.144869 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:54.144869 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:54.144869 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:54.144869 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 19:53:54.144869 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:54.144869 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 23 19:53:54.144869 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:54.144869 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 384/767] 10.222.0.13 () {56 vars in 1105 bytes} [Sun Oct 23 19:53:53 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:53:54.149571 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cca1ad-8487-4c89-b3c5-67791fbb0496 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:54.154868 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76432a5-1074-461c-831d-ef626c6434e1 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:54.194003 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2714a8d9-0fd0-4873-b56b-3a922823f56e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:54.194644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714a8d9-0fd0-4873-b56b-3a922823f56e None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:54.195237 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714a8d9-0fd0-4873-b56b-3a922823f56e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:54.195513 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714a8d9-0fd0-4873-b56b-3a922823f56e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:54.196321 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2714a8d9-0fd0-4873-b56b-3a922823f56e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:54.197683 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2714a8d9-0fd0-4873-b56b-3a922823f56e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:54.201098 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:54.219526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 384/768] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 19:53:54 2022] POST /identity/v3/endpoints => generated 334 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 19:53:54.265172 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cca1ad-8487-4c89-b3c5-67791fbb0496 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:54.310305 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe752fc-7dce-4942-86a1-813a1e8baebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:54.310940 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe752fc-7dce-4942-86a1-813a1e8baebe None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:54.311220 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe752fc-7dce-4942-86a1-813a1e8baebe None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:54.311568 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe752fc-7dce-4942-86a1-813a1e8baebe None admin] PATH_INFO: `/v3/services` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:54.312216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe752fc-7dce-4942-86a1-813a1e8baebe None admin] RBAC: Adding query filter params (name=image) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:54.312521 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe752fc-7dce-4942-86a1-813a1e8baebe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:54.313906 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe752fc-7dce-4942-86a1-813a1e8baebe None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:54.322410 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 385/769] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:54 2022] GET /identity/v3/services?name=image => generated 122 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:53:54.325721 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2714a8d9-0fd0-4873-b56b-3a922823f56e None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:54.434825 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:54.439981 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2714a8d9-0fd0-4873-b56b-3a922823f56e None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:54.478880 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b45769-4472-4508-90f5-3841e40e8dd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:54.479747 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b45769-4472-4508-90f5-3841e40e8dd1 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:54.480034 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b45769-4472-4508-90f5-3841e40e8dd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:54.480310 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b45769-4472-4508-90f5-3841e40e8dd1 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:54.480964 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b45769-4472-4508-90f5-3841e40e8dd1 None admin] RBAC: Adding query filter params (type=image) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:54.481254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b45769-4472-4508-90f5-3841e40e8dd1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:54.482823 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b45769-4472-4508-90f5-3841e40e8dd1 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:54.488931 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 385/770] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:54 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:54.495070 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe752fc-7dce-4942-86a1-813a1e8baebe None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:54.602098 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe752fc-7dce-4942-86a1-813a1e8baebe None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:54.611198 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:54.620990 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515cba37-f0de-4750-9098-85b96894a33f None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:54.621283 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515cba37-f0de-4750-9098-85b96894a33f None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:54.621568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515cba37-f0de-4750-9098-85b96894a33f None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:54.622961 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 386/771] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 19:53:54.628533 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9c2de-45a5-43dd-b7f4-90ad389f757b None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:54.628817 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9c2de-45a5-43dd-b7f4-90ad389f757b None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:54.629097 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a9c2de-45a5-43dd-b7f4-90ad389f757b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:54.642572 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3c32b-a724-4df4-801e-0b28652be28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:54.643190 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3c32b-a724-4df4-801e-0b28652be28b None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:54.643472 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3c32b-a724-4df4-801e-0b28652be28b None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:54.643748 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb3c32b-a724-4df4-801e-0b28652be28b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:54.644516 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb3c32b-a724-4df4-801e-0b28652be28b None admin] RBAC: Adding query filter params (interface=public, service_id=d90869136b1c49d68562810710f6ae5d, region_id=RegionOne) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:54.644762 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb3c32b-a724-4df4-801e-0b28652be28b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:54.646130 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb3c32b-a724-4df4-801e-0b28652be28b None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:54.648757 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-f8a9c2de-45a5-43dd-b7f4-90ad389f757b None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:54.650368 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f8a9c2de-45a5-43dd-b7f4-90ad389f757b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:54.651747 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 386/772] 10.222.0.13 () {56 vars in 1251 bytes} [Sun Oct 23 19:53:54 2022] GET /identity/v3/endpoints?service_id=d90869136b1c49d68562810710f6ae5d&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:54.657058 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3c32b-a724-4df4-801e-0b28652be28b None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:54.736140 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 387/773] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:54 2022] POST /identity/v3/auth/tokens => generated 1822 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 19:53:54.759201 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb3c32b-a724-4df4-801e-0b28652be28b None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:54.793153 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5db2f5-99d9-4a32-adbe-ca4ac22a99e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:54.793768 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5db2f5-99d9-4a32-adbe-ca4ac22a99e0 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:54.794050 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5db2f5-99d9-4a32-adbe-ca4ac22a99e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:54.794350 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5db2f5-99d9-4a32-adbe-ca4ac22a99e0 None admin] PATH_INFO: `/v3/services` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:54.794961 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5db2f5-99d9-4a32-adbe-ca4ac22a99e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:54.796870 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5db2f5-99d9-4a32-adbe-ca4ac22a99e0 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:54.810346 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 387/774] 10.222.0.13 () {56 vars in 1087 bytes} [Sun Oct 23 19:53:54 2022] GET /identity/v3/services => generated 1957 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 19:53:55.001102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a9c2de-45a5-43dd-b7f4-90ad389f757b None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:55.097846 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:55.107473 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8a9c2de-45a5-43dd-b7f4-90ad389f757b None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:55.142315 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f00a368-ec3f-4aa7-9193-3263c3a34bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:55.142954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00a368-ec3f-4aa7-9193-3263c3a34bee None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:55.143227 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00a368-ec3f-4aa7-9193-3263c3a34bee None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:55.143496 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00a368-ec3f-4aa7-9193-3263c3a34bee None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:55.144078 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f00a368-ec3f-4aa7-9193-3263c3a34bee None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:55.145421 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f00a368-ec3f-4aa7-9193-3263c3a34bee None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:55.154286 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-3f00a368-ec3f-4aa7-9193-3263c3a34bee None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 19:53:55.154286 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:55.154286 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 19:53:55.154286 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 19:53:55.154286 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 19:53:55.154286 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 19:53:55.154286 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 19:53:55.154286 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:55.154286 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 19:53:55.154286 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:55.154286 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 19:53:55.154286 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:55.154286 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:55.154286 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:55.154286 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:55.154286 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:55.154286 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:55.155749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:55.155749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:55.155749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:55.155749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:55.155749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:55.155749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:55.155749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 19:53:55.155749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 19:53:55.155749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 19:53:55.155749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 19:53:55.155749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:55.155749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:55.155749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:55.155749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:55.155749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:55.155749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:55.157008 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:55.157008 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:55.157008 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:55.157008 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:55.157008 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:55.157008 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:55.157008 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:55.157008 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:55.157008 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:55.157008 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:55.157008 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 19:53:55.157008 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:55.157008 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 19:53:55.157008 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:55.158427 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 388/775] 10.222.0.13 () {56 vars in 1126 bytes} [Sun Oct 23 19:53:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:55.162217 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5db2f5-99d9-4a32-adbe-ca4ac22a99e0 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:55.198472 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:55.266072 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5db2f5-99d9-4a32-adbe-ca4ac22a99e0 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:55.299224 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2feb389-5040-4d14-922a-fe0870b401ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:55.299829 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2feb389-5040-4d14-922a-fe0870b401ec None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:55.300105 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2feb389-5040-4d14-922a-fe0870b401ec None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:55.300374 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2feb389-5040-4d14-922a-fe0870b401ec None admin] PATH_INFO: `/v3/services` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:55.301000 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2feb389-5040-4d14-922a-fe0870b401ec None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:55.301296 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2feb389-5040-4d14-922a-fe0870b401ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:55.302766 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2feb389-5040-4d14-922a-fe0870b401ec None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:55.312729 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 388/776] 10.222.0.13 () {56 vars in 1122 bytes} [Sun Oct 23 19:53:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:55.314513 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f00a368-ec3f-4aa7-9193-3263c3a34bee None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:55.415694 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f00a368-ec3f-4aa7-9193-3263c3a34bee None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:55.449409 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dfb14f2-8e7f-4d24-85be-1e61e23b486b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:55.450018 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb14f2-8e7f-4d24-85be-1e61e23b486b None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:55.450330 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb14f2-8e7f-4d24-85be-1e61e23b486b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:55.450612 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfb14f2-8e7f-4d24-85be-1e61e23b486b None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:55.451249 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb14f2-8e7f-4d24-85be-1e61e23b486b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:55.451552 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb14f2-8e7f-4d24-85be-1e61e23b486b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:55.452987 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dfb14f2-8e7f-4d24-85be-1e61e23b486b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:55.462647 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 389/777] 10.222.0.13 () {56 vars in 1122 bytes} [Sun Oct 23 19:53:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:55.463852 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2feb389-5040-4d14-922a-fe0870b401ec None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:55.565811 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:55.569620 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2feb389-5040-4d14-922a-fe0870b401ec None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:55.604526 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c1ce07-de6c-452f-ad28-b3f59dc2e5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:55.605137 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c1ce07-de6c-452f-ad28-b3f59dc2e5c4 None admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:55.605414 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c1ce07-de6c-452f-ad28-b3f59dc2e5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:55.605682 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c1ce07-de6c-452f-ad28-b3f59dc2e5c4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:55.606383 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c1ce07-de6c-452f-ad28-b3f59dc2e5c4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:55.607748 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c1ce07-de6c-452f-ad28-b3f59dc2e5c4 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:55.634283 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 389/778] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 19:53:55 2022] POST /identity/v3/endpoints => generated 325 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:53:56.472533 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab2666-7d26-478c-a0fa-fb44d1469926 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:56.473072 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab2666-7d26-478c-a0fa-fb44d1469926 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:56.473072 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faab2666-7d26-478c-a0fa-fb44d1469926 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:56.474597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 390/779] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:56.479494 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f17231-c6bf-434f-91e9-f84c3f1965d4 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:56.479786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f17231-c6bf-434f-91e9-f84c3f1965d4 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:56.480062 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f17231-c6bf-434f-91e9-f84c3f1965d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:56.500182 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-b2f17231-c6bf-434f-91e9-f84c3f1965d4 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:56.501380 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-b2f17231-c6bf-434f-91e9-f84c3f1965d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:56.578290 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 390/780] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:56 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:53:56.848107 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab2666-7d26-478c-a0fa-fb44d1469926 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:56.870486 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:56.937142 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e50306-3f48-4ad0-a97e-638669ce1cfd None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:56.937689 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e50306-3f48-4ad0-a97e-638669ce1cfd None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:56.937689 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e50306-3f48-4ad0-a97e-638669ce1cfd None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:56.939560 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 391/781] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 19:53:56.945074 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42945761-6c46-4b44-99c1-d961f50a7b82 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:56.945355 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42945761-6c46-4b44-99c1-d961f50a7b82 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:56.945628 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42945761-6c46-4b44-99c1-d961f50a7b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:56.958571 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faab2666-7d26-478c-a0fa-fb44d1469926 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:56.963184 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-42945761-6c46-4b44-99c1-d961f50a7b82 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:56.964352 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-42945761-6c46-4b44-99c1-d961f50a7b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:56.993804 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb0128-e948-4bd3-a86a-8ebe1a17309a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:56.994461 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb0128-e948-4bd3-a86a-8ebe1a17309a None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:56.994737 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb0128-e948-4bd3-a86a-8ebe1a17309a None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:56.995010 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eb0128-e948-4bd3-a86a-8ebe1a17309a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:57.007087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eb0128-e948-4bd3-a86a-8ebe1a17309a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:53:57.011887 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-97eb0128-e948-4bd3-a86a-8ebe1a17309a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 23 19:53:57.011887 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:57.011887 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:57.011887 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:57.011887 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:57.011887 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:57.011887 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:57.011887 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:57.011887 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:57.011887 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:57.011887 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:57.011887 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:57.011887 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:53:57.011887 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:53:57.011887 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:53:57.011887 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:53:57.012904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:53:57.012904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:53:57.012904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:53:57.012904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:53:57.012904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:57.012904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:57.012904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:57.012904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:57.012904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:57.012904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:57.012904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:57.012904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:57.012904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:57.012904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:57.012904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:57.012904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:57.015183 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:57.015183 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:57.015183 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:57.015183 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:57.015183 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:53:57.015183 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:53:57.015183 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:53:57.015183 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:53:57.015183 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:53:57.015183 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:53:57.015183 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 23 19:53:57.015183 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:57.015183 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 391/782] 10.222.0.13 () {56 vars in 1120 bytes} [Sun Oct 23 19:53:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:57.018820 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb0128-e948-4bd3-a86a-8ebe1a17309a None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:57.043882 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 392/783] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:56 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 19:53:57.100597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:57.120684 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eb0128-e948-4bd3-a86a-8ebe1a17309a None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:57.153486 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3ac3b-ae09-4d55-bfd9-62f9c3f292e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:57.154086 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3ac3b-ae09-4d55-bfd9-62f9c3f292e1 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:57.154393 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3ac3b-ae09-4d55-bfd9-62f9c3f292e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:57.154750 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3ac3b-ae09-4d55-bfd9-62f9c3f292e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:57.155965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc3ac3b-ae09-4d55-bfd9-62f9c3f292e1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:57.156265 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc3ac3b-ae09-4d55-bfd9-62f9c3f292e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:57.157605 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc3ac3b-ae09-4d55-bfd9-62f9c3f292e1 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:57.169733 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 392/784] 10.222.0.13 () {56 vars in 1115 bytes} [Sun Oct 23 19:53:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:53:57.171114 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42945761-6c46-4b44-99c1-d961f50a7b82 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:57.188838 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:57.272632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42945761-6c46-4b44-99c1-d961f50a7b82 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:57.306189 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9738097e-fdfe-4767-883d-5ebaca455903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:57.306841 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738097e-fdfe-4767-883d-5ebaca455903 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:57.307115 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738097e-fdfe-4767-883d-5ebaca455903 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:57.307383 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738097e-fdfe-4767-883d-5ebaca455903 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:57.311623 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3ac3b-ae09-4d55-bfd9-62f9c3f292e1 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:57.318077 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9738097e-fdfe-4767-883d-5ebaca455903 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:57.319539 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9738097e-fdfe-4767-883d-5ebaca455903 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:57.336638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-9738097e-fdfe-4767-883d-5ebaca455903 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:57.336638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:57.336638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:57.336638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:57.336638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:57.336638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:57.336638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:57.336638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:57.336638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:57.336638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:57.336638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:57.336638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:57.336638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:57.336638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:57.336638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:57.336638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:57.337630 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:57.337630 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:57.337630 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:57.337630 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:57.337630 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:57.337630 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:57.337630 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:57.337630 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:57.337630 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:57.337630 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:57.337630 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:57.337630 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:57.337630 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:57.337630 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:57.337630 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:57.337630 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:57.337630 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:57.338569 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:57.338569 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:57.338569 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:57.338569 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 393/785] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:53:57.343906 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9738097e-fdfe-4767-883d-5ebaca455903 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:57.415543 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:57.423693 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3ac3b-ae09-4d55-bfd9-62f9c3f292e1 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:57.452129 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9738097e-fdfe-4767-883d-5ebaca455903 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:57.459014 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c87e0-c01e-4baa-888c-7273c72d15c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:57.460388 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c87e0-c01e-4baa-888c-7273c72d15c3 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:57.460856 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c87e0-c01e-4baa-888c-7273c72d15c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:57.461753 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768c87e0-c01e-4baa-888c-7273c72d15c3 None admin] PATH_INFO: `/v3/services/image` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:57.462565 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768c87e0-c01e-4baa-888c-7273c72d15c3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:57.464767 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768c87e0-c01e-4baa-888c-7273c72d15c3 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:57.477426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-768c87e0-c01e-4baa-888c-7273c72d15c3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 23 19:53:57.477426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:57.477426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 19:53:57.477426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 19:53:57.477426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 19:53:57.477426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 19:53:57.477426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 19:53:57.477426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:57.477426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 23 19:53:57.477426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:57.477426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 19:53:57.477426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:57.477426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:57.477426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:57.477426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:57.477426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:57.477426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:57.478902 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:57.478902 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:57.478902 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:57.478902 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:57.478902 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:57.478902 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:57.478902 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 19:53:57.478902 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 19:53:57.478902 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 19:53:57.478902 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 19:53:57.478902 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:57.478902 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:57.478902 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:57.478902 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:57.478902 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:57.478902 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:57.479749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:57.479749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:57.479749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:57.479749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:57.479749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:57.479749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:57.479749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:57.479749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:57.479749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:57.479749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:57.479749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 19:53:57.479749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:53:57.479749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 23 19:53:57.479749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:57.480488 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 393/786] 10.222.0.13 () {56 vars in 1105 bytes} [Sun Oct 23 19:53:57 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:53:57.485243 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c87e0-c01e-4baa-888c-7273c72d15c3 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:57.492077 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a30b3-0a61-4b9a-8e95-bee97b9dc146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:57.492687 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a30b3-0a61-4b9a-8e95-bee97b9dc146 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:57.492962 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a30b3-0a61-4b9a-8e95-bee97b9dc146 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:57.493231 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a30b3-0a61-4b9a-8e95-bee97b9dc146 None admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:57.493867 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3a30b3-0a61-4b9a-8e95-bee97b9dc146 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:57.494384 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:57.494596 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3a30b3-0a61-4b9a-8e95-bee97b9dc146 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:57.495995 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3a30b3-0a61-4b9a-8e95-bee97b9dc146 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:57.509357 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 394/787] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:53:57.514997 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a30b3-0a61-4b9a-8e95-bee97b9dc146 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:57.589849 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768c87e0-c01e-4baa-888c-7273c72d15c3 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:57.619460 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a30b3-0a61-4b9a-8e95-bee97b9dc146 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:57.627838 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918671c-1dc6-496f-88a6-90388a5029e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:57.628264 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918671c-1dc6-496f-88a6-90388a5029e6 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:57.628549 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918671c-1dc6-496f-88a6-90388a5029e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:57.628828 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c918671c-1dc6-496f-88a6-90388a5029e6 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:57.629460 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c918671c-1dc6-496f-88a6-90388a5029e6 None admin] RBAC: Adding query filter params (name=image) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:57.630039 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c918671c-1dc6-496f-88a6-90388a5029e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:57.631145 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c918671c-1dc6-496f-88a6-90388a5029e6 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:57.640773 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 394/788] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:53:57.641688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918671c-1dc6-496f-88a6-90388a5029e6 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:57.652729 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:57.661480 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce250cd-44d7-4239-b48e-0cc9f22922ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:57.662166 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce250cd-44d7-4239-b48e-0cc9f22922ba None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:57.662472 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce250cd-44d7-4239-b48e-0cc9f22922ba None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:57.662766 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce250cd-44d7-4239-b48e-0cc9f22922ba None admin] PATH_INFO: `/v3/users/nova` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:57.670101 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce250cd-44d7-4239-b48e-0cc9f22922ba None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:57.671671 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce250cd-44d7-4239-b48e-0cc9f22922ba None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:57.683636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-fce250cd-44d7-4239-b48e-0cc9f22922ba None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 23 19:53:57.683636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:57.683636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:57.683636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:57.683636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:57.683636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:57.683636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:57.683636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:57.683636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:57.683636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:57.683636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:57.683636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:57.683636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 19:53:57.683636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 19:53:57.683636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 19:53:57.683636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 19:53:57.684703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:57.684703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:57.684703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 19:53:57.684703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:57.684703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 19:53:57.684703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:53:57.684703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:57.684703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:57.684703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:57.684703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:57.684703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:57.684703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:57.684703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:57.684703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:57.684703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:57.684703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:57.684703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:57.684703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:57.685669 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:57.685669 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:57.685669 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 19:53:57.685669 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 19:53:57.685669 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 19:53:57.685669 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 19:53:57.685669 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 19:53:57.685669 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 19:53:57.685669 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 23 19:53:57.685669 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:57.690679 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 395/789] 10.222.0.13 () {56 vars in 1093 bytes} [Sun Oct 23 19:53:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:53:57.691658 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce250cd-44d7-4239-b48e-0cc9f22922ba None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:57.721081 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:57.762023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c918671c-1dc6-496f-88a6-90388a5029e6 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:57.798428 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a3e0b0-bb7a-4e17-8101-92774668076b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:57.799342 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a3e0b0-bb7a-4e17-8101-92774668076b None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:57.799629 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a3e0b0-bb7a-4e17-8101-92774668076b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:57.800043 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a3e0b0-bb7a-4e17-8101-92774668076b None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:57.800684 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a3e0b0-bb7a-4e17-8101-92774668076b None admin] RBAC: Adding query filter params (type=image) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:57.800982 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a3e0b0-bb7a-4e17-8101-92774668076b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:57.802012 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce250cd-44d7-4239-b48e-0cc9f22922ba None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:57.802433 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a3e0b0-bb7a-4e17-8101-92774668076b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:57.807542 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 395/790] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:57 2022] GET /identity/v3/services?type=image => generated 350 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:53:57.813425 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd05b29-8983-4e60-8bf5-a539791acda8 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:57.813706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd05b29-8983-4e60-8bf5-a539791acda8 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:57.813975 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd05b29-8983-4e60-8bf5-a539791acda8 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:57.815512 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 396/791] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:53:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:53:57.816176 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd05b29-8983-4e60-8bf5-a539791acda8 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:57.837382 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd4017a-be8f-400f-bf01-eaedef4cbaaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:57.837995 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4017a-be8f-400f-bf01-eaedef4cbaaa None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:57.838305 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4017a-be8f-400f-bf01-eaedef4cbaaa None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:57.838586 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd4017a-be8f-400f-bf01-eaedef4cbaaa None admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:57.839319 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd4017a-be8f-400f-bf01-eaedef4cbaaa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:57.839627 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd4017a-be8f-400f-bf01-eaedef4cbaaa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:57.841052 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd4017a-be8f-400f-bf01-eaedef4cbaaa None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:57.858338 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 396/792] 10.222.0.13 () {56 vars in 1133 bytes} [Sun Oct 23 19:53:57 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:57.859529 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b836de-4789-43cd-bac8-02a6b0f08c3a None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:57.859816 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b836de-4789-43cd-bac8-02a6b0f08c3a None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:57.860092 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b836de-4789-43cd-bac8-02a6b0f08c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:57.876912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-14b836de-4789-43cd-bac8-02a6b0f08c3a None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:53:57.878099 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-14b836de-4789-43cd-bac8-02a6b0f08c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:53:57.920228 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:57.922215 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd05b29-8983-4e60-8bf5-a539791acda8 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:57.944938 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 397/793] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:53:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:53:57.945767 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b836de-4789-43cd-bac8-02a6b0f08c3a None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:57.957416 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef228184-80fa-4511-8495-a980d0202b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:57.958019 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef228184-80fa-4511-8495-a980d0202b9e None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:57.958325 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef228184-80fa-4511-8495-a980d0202b9e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:57.958601 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef228184-80fa-4511-8495-a980d0202b9e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:57.959259 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef228184-80fa-4511-8495-a980d0202b9e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:57.960614 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef228184-80fa-4511-8495-a980d0202b9e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:57.965100 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:57.983726 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 397/794] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 19:53:57 2022] POST /identity/v3/endpoints => generated 327 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:53:58.067872 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b836de-4789-43cd-bac8-02a6b0f08c3a None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:58.113474 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776fe271-6013-4718-96e8-e39b042b1795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:58.114129 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fe271-6013-4718-96e8-e39b042b1795 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:58.114439 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fe271-6013-4718-96e8-e39b042b1795 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:58.114712 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776fe271-6013-4718-96e8-e39b042b1795 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:58.126150 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776fe271-6013-4718-96e8-e39b042b1795 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:58.127662 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776fe271-6013-4718-96e8-e39b042b1795 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:58.144128 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-776fe271-6013-4718-96e8-e39b042b1795 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:58.144128 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:58.144128 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:58.144128 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:58.144128 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:58.144128 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:58.144128 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:58.144128 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:58.144128 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:58.144128 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:58.144128 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:58.144128 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:58.144128 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:53:58.144128 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:53:58.144128 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:53:58.144128 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:53:58.145132 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:58.145132 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:58.145132 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:58.145132 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:58.145132 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:58.145132 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:58.145132 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:58.145132 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:58.145132 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:58.145132 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:58.145132 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:58.145132 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:58.145132 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:58.145132 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:58.145132 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:58.145132 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:58.145132 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:53:58.151172 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:53:58.151172 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:53:58.151172 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:53:58.151172 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 398/795] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:58.152979 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef228184-80fa-4511-8495-a980d0202b9e None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:58.220450 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776fe271-6013-4718-96e8-e39b042b1795 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:58.247996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:58.268476 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef228184-80fa-4511-8495-a980d0202b9e None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:58.305766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c73b75-2342-4af3-86fd-430baa1bc3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:58.306412 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c73b75-2342-4af3-86fd-430baa1bc3b3 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:58.306696 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c73b75-2342-4af3-86fd-430baa1bc3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:58.306970 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c73b75-2342-4af3-86fd-430baa1bc3b3 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:58.307623 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c73b75-2342-4af3-86fd-430baa1bc3b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:58.307914 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c73b75-2342-4af3-86fd-430baa1bc3b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:58.309272 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c73b75-2342-4af3-86fd-430baa1bc3b3 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:58.322646 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 398/796] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:53:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:53:58.327897 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c73b75-2342-4af3-86fd-430baa1bc3b3 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:58.331696 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776fe271-6013-4718-96e8-e39b042b1795 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:58.365962 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b1abba-4eef-4491-9554-e44254edfdf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:58.366616 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1abba-4eef-4491-9554-e44254edfdf8 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:58.366891 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1abba-4eef-4491-9554-e44254edfdf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:58.367157 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b1abba-4eef-4491-9554-e44254edfdf8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:58.368245 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b1abba-4eef-4491-9554-e44254edfdf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:58.369667 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b1abba-4eef-4491-9554-e44254edfdf8 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:58.374380 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 399/797] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:53:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:53:58.377459 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b1abba-4eef-4491-9554-e44254edfdf8 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:58.383550 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:58.437193 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c73b75-2342-4af3-86fd-430baa1bc3b3 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:58.473988 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b6394-dbb0-46cd-8b18-da0bfc684ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:58.474573 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b6394-dbb0-46cd-8b18-da0bfc684ec4 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:58.474855 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b6394-dbb0-46cd-8b18-da0bfc684ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:58.475132 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b6394-dbb0-46cd-8b18-da0bfc684ec4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:58.477283 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:58.482653 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b6394-dbb0-46cd-8b18-da0bfc684ec4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:58.484162 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b6394-dbb0-46cd-8b18-da0bfc684ec4 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:58.488101 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b1abba-4eef-4491-9554-e44254edfdf8 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:58.495738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-0c7b6394-dbb0-46cd-8b18-da0bfc684ec4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:58.495738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:53:58.495738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:53:58.495738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:53:58.495738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:53:58.495738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:53:58.495738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:53:58.495738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:53:58.495738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:53:58.495738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:53:58.495738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:53:58.495738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:53:58.495738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 19:53:58.495738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 19:53:58.495738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 19:53:58.496663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 19:53:58.496663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:53:58.496663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:53:58.496663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:53:58.496663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:53:58.496663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:53:58.496663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:53:58.496663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:53:58.496663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:53:58.496663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:53:58.496663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:53:58.496663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:53:58.496663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:53:58.496663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:53:58.496663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:53:58.496663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:53:58.496663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:53:58.497555 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 19:53:58.497555 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 19:53:58.497555 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 19:53:58.497555 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 19:53:58.497555 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 19:53:58.497555 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 19:53:58.497555 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:53:58.497555 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:53:58.497555 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 399/798] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:53:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:53:58.504429 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b6394-dbb0-46cd-8b18-da0bfc684ec4 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:58.527468 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4722bd-83f6-4ec9-9348-a783e10d6f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:58.528075 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4722bd-83f6-4ec9-9348-a783e10d6f2f None admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:58.528356 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4722bd-83f6-4ec9-9348-a783e10d6f2f None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:58.528624 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4722bd-83f6-4ec9-9348-a783e10d6f2f None admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:58.529266 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4722bd-83f6-4ec9-9348-a783e10d6f2f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:58.530764 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4722bd-83f6-4ec9-9348-a783e10d6f2f None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:58.559261 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 400/799] 10.222.0.13 () {60 vars in 1141 bytes} [Sun Oct 23 19:53:58 2022] POST /identity/v3/projects => generated 311 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 19:53:58.627937 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b6394-dbb0-46cd-8b18-da0bfc684ec4 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:58.641214 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:53:58.668429 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eac23c4-334b-48ed-8f6b-d6d90c3fd1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:58.669441 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac23c4-334b-48ed-8f6b-d6d90c3fd1f5 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:58.669905 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac23c4-334b-48ed-8f6b-d6d90c3fd1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:58.670400 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eac23c4-334b-48ed-8f6b-d6d90c3fd1f5 None admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:58.671344 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eac23c4-334b-48ed-8f6b-d6d90c3fd1f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:53:58.671834 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eac23c4-334b-48ed-8f6b-d6d90c3fd1f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:58.673479 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eac23c4-334b-48ed-8f6b-d6d90c3fd1f5 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:58.687905 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 400/800] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Oct 23 19:53:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:53:58.693982 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4722bd-83f6-4ec9-9348-a783e10d6f2f None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:53:58.802950 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4722bd-83f6-4ec9-9348-a783e10d6f2f None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:53:58.837955 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319fb515-66be-4030-bd89-a7a75a6e10c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:53:58.838974 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fb515-66be-4030-bd89-a7a75a6e10c9 None admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:53:58.839433 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fb515-66be-4030-bd89-a7a75a6e10c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:53:58.839887 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319fb515-66be-4030-bd89-a7a75a6e10c9 None admin] PATH_INFO: `/v3/projects/d40947427f5f43e7b6c82e3605737767/users/0b3d512304254ffcaa8dad90abf1f9c1/roles/032a470a172e4fe7ab796c102ceabf46` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:53:58.842169 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319fb515-66be-4030-bd89-a7a75a6e10c9 None admin] RBAC: Authorizing `identity:create_grant(project_id=d40947427f5f43e7b6c82e3605737767, user_id=0b3d512304254ffcaa8dad90abf1f9c1, role_id=032a470a172e4fe7ab796c102ceabf46)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:53:58.843875 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319fb515-66be-4030-bd89-a7a75a6e10c9 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:53:58.858966 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 401/801] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Oct 23 19:53:58 2022] PUT /identity/v3/projects/d40947427f5f43e7b6c82e3605737767/users/0b3d512304254ffcaa8dad90abf1f9c1/roles/032a470a172e4fe7ab796c102ceabf46 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 19:54:00.218176 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98691e9a-725a-4331-97c9-6c6d9889e79e None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:00.218776 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98691e9a-725a-4331-97c9-6c6d9889e79e None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:00.218776 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98691e9a-725a-4331-97c9-6c6d9889e79e None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:00.220302 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 401/802] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:54:00.225524 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51045b-4bb3-4f77-8b3f-db6a853a2b05 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:00.225903 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51045b-4bb3-4f77-8b3f-db6a853a2b05 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:00.226069 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da51045b-4bb3-4f77-8b3f-db6a853a2b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:00.233917 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:00.246117 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-da51045b-4bb3-4f77-8b3f-db6a853a2b05 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:54:00.247571 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-da51045b-4bb3-4f77-8b3f-db6a853a2b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:54:00.315717 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 402/803] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:54:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:54:00.583533 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98691e9a-725a-4331-97c9-6c6d9889e79e None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:00.691023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98691e9a-725a-4331-97c9-6c6d9889e79e None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:00.725609 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94c3d1-62f7-4c9e-880d-154262c6c8d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:00.726603 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94c3d1-62f7-4c9e-880d-154262c6c8d4 None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:00.727089 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94c3d1-62f7-4c9e-880d-154262c6c8d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:00.727582 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94c3d1-62f7-4c9e-880d-154262c6c8d4 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:00.728716 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed94c3d1-62f7-4c9e-880d-154262c6c8d4 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:00.730350 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed94c3d1-62f7-4c9e-880d-154262c6c8d4 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:00.739530 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.sql.core [None req-ed94c3d1-62f7-4c9e-880d-154262c6c8d4 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 23 19:54:00.739530 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 23 19:54:00.739530 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [parameters: {'id': 'a9d05d07bf9d4c3a8def4a67357dce40', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 23 19:54:00.739530 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81411) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 23 19:54:00.749309 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-ed94c3d1-62f7-4c9e-880d-154262c6c8d4 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 23 19:54:00.749309 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:54:00.749309 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 23 19:54:00.749309 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 23 19:54:00.749309 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 23 19:54:00.749309 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 23 19:54:00.749309 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 23 19:54:00.749309 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application result = self._query(query) Oct 23 19:54:00.749309 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 23 19:54:00.749309 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application conn.query(q) Oct 23 19:54:00.749309 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 23 19:54:00.749309 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 23 19:54:00.749309 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 23 19:54:00.749309 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application result.read() Oct 23 19:54:00.750123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 23 19:54:00.750123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 23 19:54:00.750123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 23 19:54:00.750123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 23 19:54:00.750123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 23 19:54:00.750123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 23 19:54:00.750123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 23 19:54:00.750123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 23 19:54:00.750123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 23 19:54:00.750123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:54:00.750123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 23 19:54:00.750123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:54:00.750123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:54:00.750123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 23 19:54:00.750123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 23 19:54:00.750123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 23 19:54:00.750123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 23 19:54:00.750123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 23 19:54:00.751164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application next(self.gen) Oct 23 19:54:00.751164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 23 19:54:00.751164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application yield resource Oct 23 19:54:00.751164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 23 19:54:00.751164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application next(self.gen) Oct 23 19:54:00.751164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 23 19:54:00.751164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self.session.rollback() Oct 23 19:54:00.751164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 23 19:54:00.751164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self.force_reraise() Oct 23 19:54:00.751164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 23 19:54:00.751164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise self.value Oct 23 19:54:00.751164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 23 19:54:00.751164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 23 19:54:00.751164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 23 19:54:00.751164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application session.commit() Oct 23 19:54:00.751164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 23 19:54:00.751164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 23 19:54:00.752017 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 23 19:54:00.752017 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._prepare_impl() Oct 23 19:54:00.752017 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 23 19:54:00.752017 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self.session.flush() Oct 23 19:54:00.752017 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 23 19:54:00.752017 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._flush(objects) Oct 23 19:54:00.752017 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 23 19:54:00.752017 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 23 19:54:00.752017 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 23 19:54:00.752017 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application compat.raise_( Oct 23 19:54:00.752017 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 23 19:54:00.752017 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception Oct 23 19:54:00.752017 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 23 19:54:00.752017 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application flush_context.execute() Oct 23 19:54:00.752017 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 23 19:54:00.752017 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rec.execute(self) Oct 23 19:54:00.752017 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 23 19:54:00.753568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 23 19:54:00.753568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 23 19:54:00.753568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 23 19:54:00.753568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 23 19:54:00.753568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 23 19:54:00.753568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 23 19:54:00.753568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 23 19:54:00.753568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 23 19:54:00.753568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 23 19:54:00.753568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 23 19:54:00.753568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 23 19:54:00.753568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 23 19:54:00.753568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 23 19:54:00.753568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 23 19:54:00.753568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 23 19:54:00.755525 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 23 19:54:00.755525 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception Oct 23 19:54:00.755525 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 23 19:54:00.755525 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 23 19:54:00.755525 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 23 19:54:00.755525 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 23 19:54:00.755525 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 23 19:54:00.755525 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application result = self._query(query) Oct 23 19:54:00.755525 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 23 19:54:00.755525 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application conn.query(q) Oct 23 19:54:00.755525 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 23 19:54:00.755525 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 23 19:54:00.755525 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 23 19:54:00.755525 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application result.read() Oct 23 19:54:00.755525 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 23 19:54:00.755525 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 23 19:54:00.755525 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 23 19:54:00.758016 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 23 19:54:00.758016 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 23 19:54:00.758016 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 23 19:54:00.758016 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 23 19:54:00.758016 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 23 19:54:00.758016 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 23 19:54:00.758016 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 23 19:54:00.758016 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [parameters: {'id': 'a9d05d07bf9d4c3a8def4a67357dce40', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 23 19:54:00.758016 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 23 19:54:00.758016 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:54:00.758016 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 19:54:00.758016 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:54:00.758016 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:54:00.758016 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 23 19:54:00.758016 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 23 19:54:00.759916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 23 19:54:00.759916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 23 19:54:00.759916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 23 19:54:00.759916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:54:00.759916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 19:54:00.759916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:54:00.759916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:54:00.759916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:54:00.759916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:54:00.759916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:54:00.759916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:54:00.759916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:54:00.759916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:54:00.759916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:54:00.759916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:54:00.759916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:54:00.759916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:54:00.761982 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 23 19:54:00.761982 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 23 19:54:00.761982 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:54:00.761982 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:54:00.761982 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 23 19:54:00.761982 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 23 19:54:00.761982 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:54:00.761982 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:54:00.761982 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 23 19:54:00.761982 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 23 19:54:00.761982 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 23 19:54:00.761982 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:54:00.761982 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 402/804] 10.222.0.13 () {60 vars in 1137 bytes} [Sun Oct 23 19:54:00 2022] POST /identity/v3/domains => generated 190 bytes in 169 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:54:00.766316 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51045b-4bb3-4f77-8b3f-db6a853a2b05 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:00.770215 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2650d-055f-47d3-be2c-c54ae2498f47 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:00.770529 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2650d-055f-47d3-be2c-c54ae2498f47 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:00.770800 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a2650d-055f-47d3-be2c-c54ae2498f47 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:00.772144 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 403/805] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:54:00.777323 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af612dfd-2e61-42d5-a5bb-ffba2b6f69e8 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:00.777605 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af612dfd-2e61-42d5-a5bb-ffba2b6f69e8 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:00.777880 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af612dfd-2e61-42d5-a5bb-ffba2b6f69e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:00.796503 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-af612dfd-2e61-42d5-a5bb-ffba2b6f69e8 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:54:00.797751 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-af612dfd-2e61-42d5-a5bb-ffba2b6f69e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:54:00.828694 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:00.876631 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 404/806] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:54:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 19:54:00.878066 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da51045b-4bb3-4f77-8b3f-db6a853a2b05 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:00.919161 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f6f9d-e8b8-41fb-a7ba-eb097b8dcd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:00.919796 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f6f9d-e8b8-41fb-a7ba-eb097b8dcd07 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:00.920077 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f6f9d-e8b8-41fb-a7ba-eb097b8dcd07 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:00.920350 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f6f9d-e8b8-41fb-a7ba-eb097b8dcd07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:00.922924 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:00.932313 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f6f9d-e8b8-41fb-a7ba-eb097b8dcd07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:00.933751 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f6f9d-e8b8-41fb-a7ba-eb097b8dcd07 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:00.948825 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-cc7f6f9d-e8b8-41fb-a7ba-eb097b8dcd07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:54:00.948825 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:54:00.948825 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:54:00.948825 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:54:00.948825 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:54:00.948825 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:54:00.948825 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:54:00.948825 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:54:00.948825 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:54:00.948825 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:54:00.948825 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:54:00.948825 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:54:00.948825 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:54:00.948825 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:54:00.948825 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:54:00.948825 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:54:00.949786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:54:00.949786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:54:00.949786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:54:00.949786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:54:00.949786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:54:00.949786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:54:00.949786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:54:00.949786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:54:00.949786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:54:00.949786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:54:00.949786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:54:00.949786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:54:00.949786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:54:00.949786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:54:00.949786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:54:00.949786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:54:00.949786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:54:00.955159 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:54:00.955159 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:54:00.955159 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:54:00.955159 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 403/807] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:54:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:54:00.957743 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af612dfd-2e61-42d5-a5bb-ffba2b6f69e8 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:00.994472 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493dc8c1-912e-4596-a373-d2276c130153 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:00.994813 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493dc8c1-912e-4596-a373-d2276c130153 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:00.995033 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493dc8c1-912e-4596-a373-d2276c130153 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:00.996544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 404/808] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:54:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:54:01.001644 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b726f-1480-4fcf-8b8f-bb5a749b009c None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:01.002097 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b726f-1480-4fcf-8b8f-bb5a749b009c None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:01.002227 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845b726f-1480-4fcf-8b8f-bb5a749b009c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:01.021040 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-845b726f-1480-4fcf-8b8f-bb5a749b009c None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:54:01.022220 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-845b726f-1480-4fcf-8b8f-bb5a749b009c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:54:01.067337 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af612dfd-2e61-42d5-a5bb-ffba2b6f69e8 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:01.092312 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 405/809] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:54:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 19:54:01.107073 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b45e02-9e02-4fb9-87c5-fdc87335aaf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:01.107687 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b45e02-9e02-4fb9-87c5-fdc87335aaf5 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:01.107966 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b45e02-9e02-4fb9-87c5-fdc87335aaf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:01.108763 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b45e02-9e02-4fb9-87c5-fdc87335aaf5 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:01.109407 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b45e02-9e02-4fb9-87c5-fdc87335aaf5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:54:01.109711 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b45e02-9e02-4fb9-87c5-fdc87335aaf5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:01.111105 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b45e02-9e02-4fb9-87c5-fdc87335aaf5 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:01.126312 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 405/810] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:54:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:54:01.156948 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b726f-1480-4fcf-8b8f-bb5a749b009c None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:01.267211 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845b726f-1480-4fcf-8b8f-bb5a749b009c None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:01.285517 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:01.305890 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0414428-e0e1-4b17-935d-73a775c2da46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:01.306785 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0414428-e0e1-4b17-935d-73a775c2da46 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:01.307059 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0414428-e0e1-4b17-935d-73a775c2da46 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:01.307326 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0414428-e0e1-4b17-935d-73a775c2da46 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:01.308417 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0414428-e0e1-4b17-935d-73a775c2da46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:01.309842 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0414428-e0e1-4b17-935d-73a775c2da46 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:01.311932 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 406/811] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:54:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:01.317651 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b45e02-9e02-4fb9-87c5-fdc87335aaf5 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:01.347081 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:01.367254 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0414428-e0e1-4b17-935d-73a775c2da46 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:01.431631 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b45e02-9e02-4fb9-87c5-fdc87335aaf5 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:01.468515 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b598b-fac9-4356-82b1-47ac18d80b05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:01.469120 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b598b-fac9-4356-82b1-47ac18d80b05 None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:01.469394 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b598b-fac9-4356-82b1-47ac18d80b05 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:01.469661 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930b598b-fac9-4356-82b1-47ac18d80b05 None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:01.470331 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b598b-fac9-4356-82b1-47ac18d80b05 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:01.471739 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930b598b-fac9-4356-82b1-47ac18d80b05 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:01.477310 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0414428-e0e1-4b17-935d-73a775c2da46 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:01.493726 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 406/812] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Oct 23 19:54:01 2022] POST /identity/v3/users => generated 285 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 19:54:01.515171 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152180fc-7c16-41f8-a911-cf2ddfe83cc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:01.515834 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152180fc-7c16-41f8-a911-cf2ddfe83cc2 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:01.516100 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152180fc-7c16-41f8-a911-cf2ddfe83cc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:01.516370 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152180fc-7c16-41f8-a911-cf2ddfe83cc2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:01.528064 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152180fc-7c16-41f8-a911-cf2ddfe83cc2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:54:01.533632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-152180fc-7c16-41f8-a911-cf2ddfe83cc2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 23 19:54:01.533632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:54:01.533632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:54:01.533632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:54:01.533632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:54:01.533632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:54:01.533632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:54:01.533632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:54:01.533632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:54:01.533632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:54:01.533632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:54:01.533632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:54:01.533632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:54:01.533632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:54:01.533632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:54:01.533632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:54:01.534761 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:54:01.534761 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:54:01.534761 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:54:01.534761 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:54:01.534761 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:54:01.534761 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:54:01.534761 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:54:01.534761 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:54:01.534761 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:54:01.534761 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:54:01.534761 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:54:01.534761 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:54:01.534761 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:54:01.534761 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:54:01.534761 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:54:01.534761 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:54:01.535728 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:54:01.535728 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:54:01.535728 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:54:01.535728 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:54:01.535728 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:54:01.535728 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:54:01.535728 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:54:01.535728 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:54:01.535728 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:54:01.535728 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:54:01.535728 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 23 19:54:01.535728 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:54:01.535728 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 407/813] 10.222.0.13 () {56 vars in 1120 bytes} [Sun Oct 23 19:54:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:54:01.541153 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b598b-fac9-4356-82b1-47ac18d80b05 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:01.627954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:01.649595 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930b598b-fac9-4356-82b1-47ac18d80b05 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:01.684668 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22739eab-25a9-449a-a395-bd75c1e837cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:01.685201 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22739eab-25a9-449a-a395-bd75c1e837cb None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:01.685483 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22739eab-25a9-449a-a395-bd75c1e837cb None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:01.685784 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22739eab-25a9-449a-a395-bd75c1e837cb None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:01.686502 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22739eab-25a9-449a-a395-bd75c1e837cb None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:54:01.686802 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22739eab-25a9-449a-a395-bd75c1e837cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:01.688153 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22739eab-25a9-449a-a395-bd75c1e837cb None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:01.697105 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 407/814] 10.222.0.13 () {56 vars in 1115 bytes} [Sun Oct 23 19:54:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:01.703794 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152180fc-7c16-41f8-a911-cf2ddfe83cc2 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:01.751025 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:01.803746 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152180fc-7c16-41f8-a911-cf2ddfe83cc2 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:01.840018 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e5a05a-f1cd-4e1a-b718-184e12fc5cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:01.840641 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5a05a-f1cd-4e1a-b718-184e12fc5cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:01.840918 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5a05a-f1cd-4e1a-b718-184e12fc5cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:01.841184 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e5a05a-f1cd-4e1a-b718-184e12fc5cd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:01.851667 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e5a05a-f1cd-4e1a-b718-184e12fc5cd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:01.853091 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e5a05a-f1cd-4e1a-b718-184e12fc5cd0 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:01.867342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-c8e5a05a-f1cd-4e1a-b718-184e12fc5cd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:54:01.867342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:54:01.867342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:54:01.867342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:54:01.867342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:54:01.867342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:54:01.867342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:54:01.867342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:54:01.867342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:54:01.867342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:54:01.867342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:54:01.867342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:54:01.867342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:54:01.867342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:54:01.867342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:54:01.867342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:54:01.868555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:54:01.868555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:54:01.868555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:54:01.868555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:54:01.868555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:54:01.868555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:54:01.868555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:54:01.868555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:54:01.868555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:54:01.868555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:54:01.868555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:54:01.868555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:54:01.868555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:54:01.868555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:54:01.868555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:54:01.868555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:54:01.868555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:54:01.869553 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:54:01.869553 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:54:01.869553 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:54:01.869553 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 408/815] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:54:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:54:01.874217 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22739eab-25a9-449a-a395-bd75c1e837cb None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:01.970354 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22739eab-25a9-449a-a395-bd75c1e837cb None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:02.003143 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4129b5e-c5b3-4bab-a996-fe23cbeda4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:02.003877 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4129b5e-c5b3-4bab-a996-fe23cbeda4bc None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:02.004153 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4129b5e-c5b3-4bab-a996-fe23cbeda4bc None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:02.004419 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4129b5e-c5b3-4bab-a996-fe23cbeda4bc None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:02.005054 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4129b5e-c5b3-4bab-a996-fe23cbeda4bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:54:02.005351 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4129b5e-c5b3-4bab-a996-fe23cbeda4bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:02.006718 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4129b5e-c5b3-4bab-a996-fe23cbeda4bc None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:02.019134 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 408/816] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:54:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:02.023545 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e5a05a-f1cd-4e1a-b718-184e12fc5cd0 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:02.116809 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:02.122860 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e5a05a-f1cd-4e1a-b718-184e12fc5cd0 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:02.157005 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f445c190-44ed-4715-9bac-f4e020fc6cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:02.157633 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445c190-44ed-4715-9bac-f4e020fc6cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:02.157917 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445c190-44ed-4715-9bac-f4e020fc6cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:02.158191 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f445c190-44ed-4715-9bac-f4e020fc6cd8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:02.165693 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f445c190-44ed-4715-9bac-f4e020fc6cd8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:02.167157 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f445c190-44ed-4715-9bac-f4e020fc6cd8 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:02.178216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-f445c190-44ed-4715-9bac-f4e020fc6cd8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 23 19:54:02.178216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:54:02.178216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:54:02.178216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:54:02.178216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:54:02.178216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:54:02.178216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:54:02.178216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:54:02.178216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:54:02.178216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:54:02.178216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:54:02.178216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:54:02.178216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 19:54:02.178216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 19:54:02.178216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 19:54:02.178216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 19:54:02.179274 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:54:02.179274 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:54:02.179274 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 19:54:02.179274 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:54:02.179274 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 19:54:02.179274 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:54:02.179274 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:54:02.179274 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:54:02.179274 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:54:02.179274 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:54:02.179274 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:54:02.179274 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:54:02.179274 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:54:02.179274 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:54:02.179274 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:54:02.179274 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:54:02.179274 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:54:02.179274 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:54:02.180605 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:54:02.180605 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:54:02.180605 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 19:54:02.180605 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 19:54:02.180605 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 19:54:02.180605 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 19:54:02.180605 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 19:54:02.180605 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 19:54:02.180605 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 23 19:54:02.180605 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:54:02.180605 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 409/817] 10.222.0.13 () {56 vars in 1093 bytes} [Sun Oct 23 19:54:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:54:02.185040 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4129b5e-c5b3-4bab-a996-fe23cbeda4bc None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:02.188519 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:02.286585 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4129b5e-c5b3-4bab-a996-fe23cbeda4bc None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:02.319368 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933884c0-92e3-4eb8-aa61-2d6539d4c84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:02.319980 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933884c0-92e3-4eb8-aa61-2d6539d4c84c None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:02.320242 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933884c0-92e3-4eb8-aa61-2d6539d4c84c None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:02.320517 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933884c0-92e3-4eb8-aa61-2d6539d4c84c None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:02.321242 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933884c0-92e3-4eb8-aa61-2d6539d4c84c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:54:02.321548 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933884c0-92e3-4eb8-aa61-2d6539d4c84c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:02.323000 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933884c0-92e3-4eb8-aa61-2d6539d4c84c None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:02.340261 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 409/818] 10.222.0.13 () {56 vars in 1133 bytes} [Sun Oct 23 19:54:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:02.346056 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f445c190-44ed-4715-9bac-f4e020fc6cd8 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:02.446770 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f445c190-44ed-4715-9bac-f4e020fc6cd8 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:02.479195 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e59878a-2cf5-4eae-8a41-6831ab767137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:02.479793 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e59878a-2cf5-4eae-8a41-6831ab767137 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:02.480070 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e59878a-2cf5-4eae-8a41-6831ab767137 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:02.480340 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e59878a-2cf5-4eae-8a41-6831ab767137 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:02.504565 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:02.551949 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e59878a-2cf5-4eae-8a41-6831ab767137 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:02.554425 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e59878a-2cf5-4eae-8a41-6831ab767137 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:02.650018 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 410/819] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Oct 23 19:54:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 23 19:54:02.656338 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933884c0-92e3-4eb8-aa61-2d6539d4c84c None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:02.713889 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:02.754270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933884c0-92e3-4eb8-aa61-2d6539d4c84c None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:02.787872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd19f2e2-af00-4f9a-86a5-4c397874256e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:02.788502 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19f2e2-af00-4f9a-86a5-4c397874256e None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:02.788783 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19f2e2-af00-4f9a-86a5-4c397874256e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:02.789057 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd19f2e2-af00-4f9a-86a5-4c397874256e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:02.799395 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd19f2e2-af00-4f9a-86a5-4c397874256e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:02.800825 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd19f2e2-af00-4f9a-86a5-4c397874256e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:02.815359 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-dd19f2e2-af00-4f9a-86a5-4c397874256e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:54:02.815359 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:54:02.815359 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:54:02.815359 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:54:02.815359 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:54:02.815359 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:54:02.815359 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:54:02.815359 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:54:02.815359 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:54:02.815359 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:54:02.815359 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:54:02.815359 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:54:02.815359 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:54:02.815359 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:54:02.815359 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:54:02.815359 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:54:02.816291 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:54:02.816291 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:54:02.816291 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:54:02.816291 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:54:02.816291 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:54:02.816291 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:54:02.816291 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:54:02.816291 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:54:02.816291 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:54:02.816291 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:54:02.816291 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:54:02.816291 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:54:02.816291 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:54:02.816291 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:54:02.816291 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:54:02.816291 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:54:02.816291 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:54:02.817699 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:54:02.817699 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:54:02.817699 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:54:02.817699 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 410/820] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:54:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:54:02.822050 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e59878a-2cf5-4eae-8a41-6831ab767137 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:02.876284 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:02.918816 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e59878a-2cf5-4eae-8a41-6831ab767137 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:02.951947 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c31c5d7-0151-430b-9ac7-8f0a0365a090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:02.952542 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31c5d7-0151-430b-9ac7-8f0a0365a090 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:02.952816 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31c5d7-0151-430b-9ac7-8f0a0365a090 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:02.953084 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c31c5d7-0151-430b-9ac7-8f0a0365a090 None admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:02.953718 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c31c5d7-0151-430b-9ac7-8f0a0365a090 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:54:02.954016 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c31c5d7-0151-430b-9ac7-8f0a0365a090 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:02.955399 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c31c5d7-0151-430b-9ac7-8f0a0365a090 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:02.966311 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 411/821] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:54:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:02.972389 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd19f2e2-af00-4f9a-86a5-4c397874256e None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:03.071552 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd19f2e2-af00-4f9a-86a5-4c397874256e None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:03.084550 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:03.105898 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddaff57-377e-4adf-b8b1-e8574255672a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:03.106551 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddaff57-377e-4adf-b8b1-e8574255672a None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:03.106829 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddaff57-377e-4adf-b8b1-e8574255672a None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:03.107096 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddaff57-377e-4adf-b8b1-e8574255672a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:03.113571 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddaff57-377e-4adf-b8b1-e8574255672a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:03.115032 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddaff57-377e-4adf-b8b1-e8574255672a None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:03.125953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-2ddaff57-377e-4adf-b8b1-e8574255672a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:54:03.125953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:54:03.125953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:54:03.125953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:54:03.125953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:54:03.125953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:54:03.125953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:54:03.125953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:54:03.125953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:54:03.125953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:54:03.125953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:54:03.125953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:54:03.125953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 19:54:03.125953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 19:54:03.125953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 19:54:03.127102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 19:54:03.127102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:54:03.127102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:54:03.127102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:54:03.127102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:54:03.127102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:54:03.127102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:54:03.127102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:54:03.127102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:54:03.127102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:54:03.127102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:54:03.127102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:54:03.127102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:54:03.127102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:54:03.127102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:54:03.127102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:54:03.127102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:54:03.127990 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 19:54:03.127990 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 19:54:03.127990 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 19:54:03.127990 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 19:54:03.127990 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 19:54:03.127990 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 19:54:03.127990 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:54:03.127990 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:54:03.127990 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 411/822] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:54:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:54:03.132912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c31c5d7-0151-430b-9ac7-8f0a0365a090 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:03.231009 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c31c5d7-0151-430b-9ac7-8f0a0365a090 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:03.246522 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:03.264869 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9527d3-90de-4e5e-97a1-7c2d7e4c4865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:03.265506 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9527d3-90de-4e5e-97a1-7c2d7e4c4865 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:03.265822 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9527d3-90de-4e5e-97a1-7c2d7e4c4865 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:03.266095 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9527d3-90de-4e5e-97a1-7c2d7e4c4865 None admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:03.266834 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9527d3-90de-4e5e-97a1-7c2d7e4c4865 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:54:03.267149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9527d3-90de-4e5e-97a1-7c2d7e4c4865 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:03.268578 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9527d3-90de-4e5e-97a1-7c2d7e4c4865 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:03.281268 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 412/823] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Oct 23 19:54:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:03.287391 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddaff57-377e-4adf-b8b1-e8574255672a None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:03.382999 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddaff57-377e-4adf-b8b1-e8574255672a None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:03.415846 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be9c2a-9a7c-4820-b154-bfbd70392c5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:03.416443 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be9c2a-9a7c-4820-b154-bfbd70392c5e None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:03.416719 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be9c2a-9a7c-4820-b154-bfbd70392c5e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:03.416988 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be9c2a-9a7c-4820-b154-bfbd70392c5e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:03.417700 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be9c2a-9a7c-4820-b154-bfbd70392c5e None admin] RBAC: Adding query filter params (role.id=032a470a172e4fe7ab796c102ceabf46, scope.project.id=d40947427f5f43e7b6c82e3605737767, user.id=0b3d512304254ffcaa8dad90abf1f9c1) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:54:03.418004 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be9c2a-9a7c-4820-b154-bfbd70392c5e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:03.419442 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9be9c2a-9a7c-4820-b154-bfbd70392c5e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:03.425125 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 412/824] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Oct 23 19:54:03 2022] GET /identity/v3/role_assignments?user.id=0b3d512304254ffcaa8dad90abf1f9c1&scope.project.id=d40947427f5f43e7b6c82e3605737767&role.id=032a470a172e4fe7ab796c102ceabf46 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:03.660531 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f260b3-ae8f-4dfc-89ee-17057fa70fcb None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:03.661026 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f260b3-ae8f-4dfc-89ee-17057fa70fcb None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:03.661026 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f260b3-ae8f-4dfc-89ee-17057fa70fcb None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:03.662371 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 413/825] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:54:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:54:03.667837 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022dc3a0-5eb8-43c1-83f9-feb27e3d7646 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:03.668127 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022dc3a0-5eb8-43c1-83f9-feb27e3d7646 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:03.668401 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022dc3a0-5eb8-43c1-83f9-feb27e3d7646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:03.686939 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-022dc3a0-5eb8-43c1-83f9-feb27e3d7646 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:54:03.688119 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-022dc3a0-5eb8-43c1-83f9-feb27e3d7646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:54:03.716961 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:03.758393 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 413/826] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:54:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:54:04.023494 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f260b3-ae8f-4dfc-89ee-17057fa70fcb None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:04.125839 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f260b3-ae8f-4dfc-89ee-17057fa70fcb None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:04.161592 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a1a0f-ec69-4312-82c1-eac69252bf97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:04.162219 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a1a0f-ec69-4312-82c1-eac69252bf97 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:04.162562 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a1a0f-ec69-4312-82c1-eac69252bf97 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:04.162836 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a1a0f-ec69-4312-82c1-eac69252bf97 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:04.172987 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6a1a0f-ec69-4312-82c1-eac69252bf97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:54:04.177873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-be6a1a0f-ec69-4312-82c1-eac69252bf97 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:54:04.177873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:54:04.177873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:54:04.177873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:54:04.177873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:54:04.177873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:54:04.177873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:54:04.177873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:54:04.177873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:54:04.177873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:54:04.177873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:54:04.177873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:54:04.177873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:54:04.177873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:54:04.177873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:54:04.177873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:54:04.178938 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:54:04.178938 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:54:04.178938 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:54:04.178938 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:54:04.178938 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:54:04.178938 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:54:04.178938 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:54:04.178938 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:54:04.178938 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:54:04.178938 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:54:04.178938 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:54:04.178938 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:54:04.178938 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:54:04.178938 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:54:04.178938 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:54:04.178938 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:54:04.180367 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:54:04.180367 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:54:04.180367 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:54:04.180367 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:54:04.180367 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:54:04.180367 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:54:04.180367 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:54:04.180367 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:54:04.180367 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:54:04.180367 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:54:04.180367 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:54:04.180367 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:54:04.180367 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 414/827] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Oct 23 19:54:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:54:04.185053 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022dc3a0-5eb8-43c1-83f9-feb27e3d7646 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:04.285437 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022dc3a0-5eb8-43c1-83f9-feb27e3d7646 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:04.318581 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff2d07-bcd1-481c-b6e3-0c3bd3683b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:04.319178 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff2d07-bcd1-481c-b6e3-0c3bd3683b40 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:04.319454 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff2d07-bcd1-481c-b6e3-0c3bd3683b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:04.319724 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff2d07-bcd1-481c-b6e3-0c3bd3683b40 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:04.320373 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff2d07-bcd1-481c-b6e3-0c3bd3683b40 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:54:04.320670 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff2d07-bcd1-481c-b6e3-0c3bd3683b40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:04.322010 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff2d07-bcd1-481c-b6e3-0c3bd3683b40 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:04.330217 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 414/828] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:54:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:54:04.336099 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a1a0f-ec69-4312-82c1-eac69252bf97 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:04.371869 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:04.434024 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a1a0f-ec69-4312-82c1-eac69252bf97 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:04.466790 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e78c39b-c4dd-49c4-805a-c26dcb4e4f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:04.467396 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78c39b-c4dd-49c4-805a-c26dcb4e4f50 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:04.468202 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78c39b-c4dd-49c4-805a-c26dcb4e4f50 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:04.468474 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e78c39b-c4dd-49c4-805a-c26dcb4e4f50 None admin] PATH_INFO: `/v3/users/cdfccf2a58fd410185a1679f4f6d3345` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:04.480976 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e78c39b-c4dd-49c4-805a-c26dcb4e4f50 None admin] RBAC: Authorizing `identity:get_user(user_id=cdfccf2a58fd410185a1679f4f6d3345)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:04.482429 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e78c39b-c4dd-49c4-805a-c26dcb4e4f50 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:04.484053 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 415/829] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:54:04 2022] GET /identity/v3/users/cdfccf2a58fd410185a1679f4f6d3345 => generated 285 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:04.489725 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff2d07-bcd1-481c-b6e3-0c3bd3683b40 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:04.518249 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:04.587499 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff2d07-bcd1-481c-b6e3-0c3bd3683b40 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:04.619821 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79d35e5-332d-4829-9825-e4a1af535a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:04.620437 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d35e5-332d-4829-9825-e4a1af535a82 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:04.620714 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d35e5-332d-4829-9825-e4a1af535a82 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:04.620985 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79d35e5-332d-4829-9825-e4a1af535a82 None admin] PATH_INFO: `/v3/projects/71ad19503c5843ceb4595ebfa1cfb8dd` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:04.622075 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d35e5-332d-4829-9825-e4a1af535a82 None admin] RBAC: Authorizing `identity:get_project(project_id=71ad19503c5843ceb4595ebfa1cfb8dd)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:04.623631 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79d35e5-332d-4829-9825-e4a1af535a82 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:04.625256 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 415/830] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Oct 23 19:54:04 2022] GET /identity/v3/projects/71ad19503c5843ceb4595ebfa1cfb8dd => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:04.631136 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e78c39b-c4dd-49c4-805a-c26dcb4e4f50 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:04.719831 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:04.726623 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e78c39b-c4dd-49c4-805a-c26dcb4e4f50 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:04.760049 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14153538-b868-4bcb-ad97-55aafc342413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:04.760690 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14153538-b868-4bcb-ad97-55aafc342413 None admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:04.760970 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14153538-b868-4bcb-ad97-55aafc342413 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:04.761243 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14153538-b868-4bcb-ad97-55aafc342413 None admin] PATH_INFO: `/v3/projects/71ad19503c5843ceb4595ebfa1cfb8dd/users/cdfccf2a58fd410185a1679f4f6d3345/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:04.763120 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14153538-b868-4bcb-ad97-55aafc342413 None admin] RBAC: Authorizing `identity:create_grant(project_id=71ad19503c5843ceb4595ebfa1cfb8dd, user_id=cdfccf2a58fd410185a1679f4f6d3345, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:04.764548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14153538-b868-4bcb-ad97-55aafc342413 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:04.776244 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 416/831] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Oct 23 19:54:04 2022] PUT /identity/v3/projects/71ad19503c5843ceb4595ebfa1cfb8dd/users/cdfccf2a58fd410185a1679f4f6d3345/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 19:54:06.867588 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15715da3-dd6e-4c62-a044-200cee0444c8 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:06.868141 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15715da3-dd6e-4c62-a044-200cee0444c8 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:06.868141 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15715da3-dd6e-4c62-a044-200cee0444c8 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:06.869476 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 416/832] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:54:06.874911 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498252d3-6d93-4499-995e-bae40dac5cf6 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:06.875345 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498252d3-6d93-4499-995e-bae40dac5cf6 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:06.875460 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498252d3-6d93-4499-995e-bae40dac5cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:06.893551 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-498252d3-6d93-4499-995e-bae40dac5cf6 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:54:06.894764 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-498252d3-6d93-4499-995e-bae40dac5cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:54:06.958603 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 417/833] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:54:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:54:07.224445 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15715da3-dd6e-4c62-a044-200cee0444c8 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:07.317023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:07.323900 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15715da3-dd6e-4c62-a044-200cee0444c8 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:07.357039 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eadc5b-b18c-4d5d-a56c-8702c44d8e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:07.357654 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eadc5b-b18c-4d5d-a56c-8702c44d8e33 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:07.357929 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eadc5b-b18c-4d5d-a56c-8702c44d8e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:07.358744 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eadc5b-b18c-4d5d-a56c-8702c44d8e33 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:07.368337 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eadc5b-b18c-4d5d-a56c-8702c44d8e33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:54:07.373087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-13eadc5b-b18c-4d5d-a56c-8702c44d8e33 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:54:07.373087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:54:07.373087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:54:07.373087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:54:07.373087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:54:07.373087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:54:07.373087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:54:07.373087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:54:07.373087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:54:07.373087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:54:07.373087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:54:07.373087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:54:07.373087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:54:07.373087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:54:07.373087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:54:07.373087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:54:07.374164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:54:07.374164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:54:07.374164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:54:07.374164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:54:07.374164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:54:07.374164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:54:07.374164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:54:07.374164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:54:07.374164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:54:07.374164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:54:07.374164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:54:07.374164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:54:07.374164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:54:07.374164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:54:07.374164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:54:07.374164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:54:07.375380 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:54:07.375380 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:54:07.375380 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:54:07.375380 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:54:07.375380 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:54:07.375380 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:54:07.375380 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:54:07.375380 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:54:07.375380 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:54:07.375380 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:54:07.375380 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:54:07.375380 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:54:07.375380 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 417/834] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Oct 23 19:54:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:54:07.380224 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498252d3-6d93-4499-995e-bae40dac5cf6 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:07.460178 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:07.477126 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498252d3-6d93-4499-995e-bae40dac5cf6 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:07.509378 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f83e44-ad28-4a3e-aaef-8e57b86c12bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:07.509976 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f83e44-ad28-4a3e-aaef-8e57b86c12bf None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:07.510268 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f83e44-ad28-4a3e-aaef-8e57b86c12bf None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:07.510547 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f83e44-ad28-4a3e-aaef-8e57b86c12bf None admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:07.511201 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f83e44-ad28-4a3e-aaef-8e57b86c12bf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:54:07.511500 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f83e44-ad28-4a3e-aaef-8e57b86c12bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:07.512837 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f83e44-ad28-4a3e-aaef-8e57b86c12bf None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:07.521050 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 418/835] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:54:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:07.526854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eadc5b-b18c-4d5d-a56c-8702c44d8e33 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:07.621894 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eadc5b-b18c-4d5d-a56c-8702c44d8e33 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:07.653885 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5394b2-a09e-4d8d-868e-73ee86c9a952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:07.654511 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5394b2-a09e-4d8d-868e-73ee86c9a952 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:07.654786 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5394b2-a09e-4d8d-868e-73ee86c9a952 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:07.655082 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5394b2-a09e-4d8d-868e-73ee86c9a952 None admin] PATH_INFO: `/v3/users/cdfccf2a58fd410185a1679f4f6d3345` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:07.656172 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5394b2-a09e-4d8d-868e-73ee86c9a952 None admin] RBAC: Authorizing `identity:get_user(user_id=cdfccf2a58fd410185a1679f4f6d3345)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:07.657589 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5394b2-a09e-4d8d-868e-73ee86c9a952 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:07.659151 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 418/836] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:54:07 2022] GET /identity/v3/users/cdfccf2a58fd410185a1679f4f6d3345 => generated 285 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:07.664566 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f83e44-ad28-4a3e-aaef-8e57b86c12bf None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:07.762198 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f83e44-ad28-4a3e-aaef-8e57b86c12bf None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:07.793796 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363dfd1-c6ad-42c8-abdf-111fdb33a6e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:07.794416 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363dfd1-c6ad-42c8-abdf-111fdb33a6e7 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:07.794699 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363dfd1-c6ad-42c8-abdf-111fdb33a6e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:07.794971 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1363dfd1-c6ad-42c8-abdf-111fdb33a6e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:07.809887 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:07.861055 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363dfd1-c6ad-42c8-abdf-111fdb33a6e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:07.862527 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1363dfd1-c6ad-42c8-abdf-111fdb33a6e7 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:07.957013 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 419/837] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Oct 23 19:54:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 23 19:54:07.963026 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5394b2-a09e-4d8d-868e-73ee86c9a952 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:07.988375 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:08.062413 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5394b2-a09e-4d8d-868e-73ee86c9a952 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:08.095375 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec17c88-50ca-48ce-a189-b24719e764e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:08.095986 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec17c88-50ca-48ce-a189-b24719e764e2 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:08.096260 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec17c88-50ca-48ce-a189-b24719e764e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:08.096529 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec17c88-50ca-48ce-a189-b24719e764e2 None admin] PATH_INFO: `/v3/projects/71ad19503c5843ceb4595ebfa1cfb8dd` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:08.097612 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec17c88-50ca-48ce-a189-b24719e764e2 None admin] RBAC: Authorizing `identity:get_project(project_id=71ad19503c5843ceb4595ebfa1cfb8dd)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:08.099038 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec17c88-50ca-48ce-a189-b24719e764e2 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:08.100634 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 419/838] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Oct 23 19:54:07 2022] GET /identity/v3/projects/71ad19503c5843ceb4595ebfa1cfb8dd => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:08.106693 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363dfd1-c6ad-42c8-abdf-111fdb33a6e7 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:08.151470 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:08.203329 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1363dfd1-c6ad-42c8-abdf-111fdb33a6e7 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:08.235311 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f06cd09-de61-4639-a274-8c5459785726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:08.235938 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06cd09-de61-4639-a274-8c5459785726 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:08.236218 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06cd09-de61-4639-a274-8c5459785726 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:08.236490 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f06cd09-de61-4639-a274-8c5459785726 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:08.237207 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f06cd09-de61-4639-a274-8c5459785726 None admin] RBAC: Adding query filter params (role.id=742dfaf63ef74207a2c9213411513925, scope.project.id=71ad19503c5843ceb4595ebfa1cfb8dd, user.id=cdfccf2a58fd410185a1679f4f6d3345) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:54:08.237515 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f06cd09-de61-4639-a274-8c5459785726 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:08.239016 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f06cd09-de61-4639-a274-8c5459785726 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:08.244573 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 420/839] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Oct 23 19:54:08 2022] GET /identity/v3/role_assignments?user.id=cdfccf2a58fd410185a1679f4f6d3345&scope.project.id=71ad19503c5843ceb4595ebfa1cfb8dd&role.id=742dfaf63ef74207a2c9213411513925 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:10.345090 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567e8f9-bbcc-4f05-8a10-1cdd2be46680 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:10.345615 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567e8f9-bbcc-4f05-8a10-1cdd2be46680 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:10.345615 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567e8f9-bbcc-4f05-8a10-1cdd2be46680 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:10.347038 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 420/840] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:54:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:54:10.352193 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291c631-64de-4f29-ab1f-c999c3da1eb5 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:10.352574 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291c631-64de-4f29-ab1f-c999c3da1eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:10.352732 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d291c631-64de-4f29-ab1f-c999c3da1eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:10.370891 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-d291c631-64de-4f29-ab1f-c999c3da1eb5 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:54:10.372055 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-d291c631-64de-4f29-ab1f-c999c3da1eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:54:10.435637 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 421/841] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:54:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:54:10.706715 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567e8f9-bbcc-4f05-8a10-1cdd2be46680 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:10.917427 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:10.927033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567e8f9-bbcc-4f05-8a10-1cdd2be46680 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:10.959598 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd10a1a2-e0a7-4752-95eb-4f4660cb3bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:10.960151 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10a1a2-e0a7-4752-95eb-4f4660cb3bd0 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:10.960426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10a1a2-e0a7-4752-95eb-4f4660cb3bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:10.960694 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd10a1a2-e0a7-4752-95eb-4f4660cb3bd0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:10.961773 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd10a1a2-e0a7-4752-95eb-4f4660cb3bd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:10.963200 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd10a1a2-e0a7-4752-95eb-4f4660cb3bd0 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:10.964748 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 421/842] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:54:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:10.971045 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291c631-64de-4f29-ab1f-c999c3da1eb5 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:11.010768 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:11.067745 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d291c631-64de-4f29-ab1f-c999c3da1eb5 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:11.100477 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc5a7f-4603-419d-99a9-314771931de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:11.101079 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc5a7f-4603-419d-99a9-314771931de5 None admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:11.101362 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc5a7f-4603-419d-99a9-314771931de5 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:11.101643 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc5a7f-4603-419d-99a9-314771931de5 None admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:11.102310 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fc5a7f-4603-419d-99a9-314771931de5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:11.103727 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fc5a7f-4603-419d-99a9-314771931de5 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:11.124561 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 422/843] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Oct 23 19:54:10 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:54:13.261032 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2044c93-23f7-4618-874f-c7e1145e65a4 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:13.261572 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2044c93-23f7-4618-874f-c7e1145e65a4 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:13.261572 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2044c93-23f7-4618-874f-c7e1145e65a4 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:13.263155 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 422/844] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:54:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:54:13.268364 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ae9a9-d967-4fb7-83c1-3c2685a3863f None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:13.268806 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ae9a9-d967-4fb7-83c1-3c2685a3863f None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:13.268899 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90ae9a9-d967-4fb7-83c1-3c2685a3863f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:13.287170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-d90ae9a9-d967-4fb7-83c1-3c2685a3863f None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:54:13.288279 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-d90ae9a9-d967-4fb7-83c1-3c2685a3863f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:54:13.351444 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 423/845] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:54:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:54:13.618027 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2044c93-23f7-4618-874f-c7e1145e65a4 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:13.717030 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2044c93-23f7-4618-874f-c7e1145e65a4 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:13.751170 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad027b4-1189-4ae3-95ce-aa0688e7bcd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:13.751791 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad027b4-1189-4ae3-95ce-aa0688e7bcd7 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:13.752068 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad027b4-1189-4ae3-95ce-aa0688e7bcd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:13.752341 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad027b4-1189-4ae3-95ce-aa0688e7bcd7 None admin] PATH_INFO: `/v3/roles/2989d29f392d4d8c802354159891eb11` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:13.753719 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad027b4-1189-4ae3-95ce-aa0688e7bcd7 None admin] RBAC: Authorizing `identity:get_role(role_id=2989d29f392d4d8c802354159891eb11)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:13.755105 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad027b4-1189-4ae3-95ce-aa0688e7bcd7 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:13.756424 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 423/846] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:54:13 2022] GET /identity/v3/roles/2989d29f392d4d8c802354159891eb11 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:13.762151 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ae9a9-d967-4fb7-83c1-3c2685a3863f None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:13.787565 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:13.857961 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90ae9a9-d967-4fb7-83c1-3c2685a3863f None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:13.891047 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0938b89-9fd2-402c-b4a2-b2349663b0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:13.891605 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0938b89-9fd2-402c-b4a2-b2349663b0f1 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:13.891879 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0938b89-9fd2-402c-b4a2-b2349663b0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:13.892149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0938b89-9fd2-402c-b4a2-b2349663b0f1 None admin] PATH_INFO: `/v3/users/ebc0fa02ab024d6d86c055001e403304` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:13.905771 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0938b89-9fd2-402c-b4a2-b2349663b0f1 None admin] RBAC: Authorizing `identity:get_user(user_id=ebc0fa02ab024d6d86c055001e403304)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:13.907217 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0938b89-9fd2-402c-b4a2-b2349663b0f1 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:13.908790 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 424/847] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:54:13 2022] GET /identity/v3/users/ebc0fa02ab024d6d86c055001e403304 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:13.914422 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad027b4-1189-4ae3-95ce-aa0688e7bcd7 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:13.943299 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:14.012220 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad027b4-1189-4ae3-95ce-aa0688e7bcd7 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:14.044844 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264db3fa-5dee-4658-bcab-df986111dcaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:14.045482 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264db3fa-5dee-4658-bcab-df986111dcaf None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:14.046322 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264db3fa-5dee-4658-bcab-df986111dcaf None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:14.046601 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264db3fa-5dee-4658-bcab-df986111dcaf None admin] PATH_INFO: `/v3/projects/71ad19503c5843ceb4595ebfa1cfb8dd` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:14.047703 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264db3fa-5dee-4658-bcab-df986111dcaf None admin] RBAC: Authorizing `identity:get_project(project_id=71ad19503c5843ceb4595ebfa1cfb8dd)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:14.049117 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264db3fa-5dee-4658-bcab-df986111dcaf None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:14.050768 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 424/848] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Oct 23 19:54:13 2022] GET /identity/v3/projects/71ad19503c5843ceb4595ebfa1cfb8dd => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:14.056598 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0938b89-9fd2-402c-b4a2-b2349663b0f1 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:14.137519 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:14.153376 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0938b89-9fd2-402c-b4a2-b2349663b0f1 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:14.185306 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f45d938-b12c-4757-9961-0aa0592113eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:14.185948 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45d938-b12c-4757-9961-0aa0592113eb None admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:14.186767 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45d938-b12c-4757-9961-0aa0592113eb None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:14.187043 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f45d938-b12c-4757-9961-0aa0592113eb None admin] PATH_INFO: `/v3/projects/71ad19503c5843ceb4595ebfa1cfb8dd/users/ebc0fa02ab024d6d86c055001e403304/roles/2989d29f392d4d8c802354159891eb11` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:14.189146 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f45d938-b12c-4757-9961-0aa0592113eb None admin] RBAC: Authorizing `identity:create_grant(project_id=71ad19503c5843ceb4595ebfa1cfb8dd, user_id=ebc0fa02ab024d6d86c055001e403304, role_id=2989d29f392d4d8c802354159891eb11)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:14.190865 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f45d938-b12c-4757-9961-0aa0592113eb None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:14.203843 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 425/849] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Oct 23 19:54:14 2022] PUT /identity/v3/projects/71ad19503c5843ceb4595ebfa1cfb8dd/users/ebc0fa02ab024d6d86c055001e403304/roles/2989d29f392d4d8c802354159891eb11 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 19:54:16.286062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116e5c2-2851-4c37-9267-260b06df0608 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:16.286602 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116e5c2-2851-4c37-9267-260b06df0608 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:16.286602 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7116e5c2-2851-4c37-9267-260b06df0608 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:16.287878 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 425/850] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:54:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:54:16.293191 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f6c82-f5f9-43ed-a0e9-3e9dfa85ae39 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:16.293624 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f6c82-f5f9-43ed-a0e9-3e9dfa85ae39 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:16.293717 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16f6c82-f5f9-43ed-a0e9-3e9dfa85ae39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:16.312071 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-c16f6c82-f5f9-43ed-a0e9-3e9dfa85ae39 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:54:16.313234 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-c16f6c82-f5f9-43ed-a0e9-3e9dfa85ae39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:54:16.376284 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 426/851] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:54:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:54:16.642145 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116e5c2-2851-4c37-9267-260b06df0608 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:16.734418 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:16.741129 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7116e5c2-2851-4c37-9267-260b06df0608 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:16.773609 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e03f2-4cc3-42e5-9d3f-55d065574971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:16.774223 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e03f2-4cc3-42e5-9d3f-55d065574971 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:16.774530 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e03f2-4cc3-42e5-9d3f-55d065574971 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:16.774799 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e03f2-4cc3-42e5-9d3f-55d065574971 None admin] PATH_INFO: `/v3/roles/2989d29f392d4d8c802354159891eb11` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:16.776155 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e03f2-4cc3-42e5-9d3f-55d065574971 None admin] RBAC: Authorizing `identity:get_role(role_id=2989d29f392d4d8c802354159891eb11)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:16.777504 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e03f2-4cc3-42e5-9d3f-55d065574971 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:16.778854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 426/852] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:54:16 2022] GET /identity/v3/roles/2989d29f392d4d8c802354159891eb11 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:16.785262 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f6c82-f5f9-43ed-a0e9-3e9dfa85ae39 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:16.856387 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:16.881001 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16f6c82-f5f9-43ed-a0e9-3e9dfa85ae39 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:16.914054 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9cb7d0-3339-4727-8c10-bc5bc42dca1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:16.914640 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9cb7d0-3339-4727-8c10-bc5bc42dca1c None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:16.914919 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9cb7d0-3339-4727-8c10-bc5bc42dca1c None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:16.915193 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9cb7d0-3339-4727-8c10-bc5bc42dca1c None admin] PATH_INFO: `/v3/users/ebc0fa02ab024d6d86c055001e403304` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:16.916290 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9cb7d0-3339-4727-8c10-bc5bc42dca1c None admin] RBAC: Authorizing `identity:get_user(user_id=ebc0fa02ab024d6d86c055001e403304)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:16.917702 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9cb7d0-3339-4727-8c10-bc5bc42dca1c None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:16.919300 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 427/853] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:54:16 2022] GET /identity/v3/users/ebc0fa02ab024d6d86c055001e403304 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:16.925141 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e03f2-4cc3-42e5-9d3f-55d065574971 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:17.022510 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e03f2-4cc3-42e5-9d3f-55d065574971 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:17.054512 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48964e-8644-42da-aa4d-22b97a46e96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:17.055105 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48964e-8644-42da-aa4d-22b97a46e96b None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:17.055381 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48964e-8644-42da-aa4d-22b97a46e96b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:17.055648 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e48964e-8644-42da-aa4d-22b97a46e96b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:17.086023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:17.120445 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e48964e-8644-42da-aa4d-22b97a46e96b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:17.121864 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e48964e-8644-42da-aa4d-22b97a46e96b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:17.214646 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 427/854] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Oct 23 19:54:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 23 19:54:17.220495 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9cb7d0-3339-4727-8c10-bc5bc42dca1c None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:17.316897 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9cb7d0-3339-4727-8c10-bc5bc42dca1c None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:17.348863 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e854594b-7505-46c7-bf7f-1c4908fed204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:17.349472 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854594b-7505-46c7-bf7f-1c4908fed204 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:17.349751 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854594b-7505-46c7-bf7f-1c4908fed204 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:17.350046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e854594b-7505-46c7-bf7f-1c4908fed204 None admin] PATH_INFO: `/v3/projects/71ad19503c5843ceb4595ebfa1cfb8dd` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:17.351169 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e854594b-7505-46c7-bf7f-1c4908fed204 None admin] RBAC: Authorizing `identity:get_project(project_id=71ad19503c5843ceb4595ebfa1cfb8dd)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:17.352586 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e854594b-7505-46c7-bf7f-1c4908fed204 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:17.354184 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 428/855] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Oct 23 19:54:17 2022] GET /identity/v3/projects/71ad19503c5843ceb4595ebfa1cfb8dd => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:17.360697 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48964e-8644-42da-aa4d-22b97a46e96b None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:17.418503 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:17.455250 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e48964e-8644-42da-aa4d-22b97a46e96b None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:17.487605 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbdb948-f9df-41d7-b5c2-306971ed544c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:17.488138 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbdb948-f9df-41d7-b5c2-306971ed544c None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:17.488413 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbdb948-f9df-41d7-b5c2-306971ed544c None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:17.488681 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbdb948-f9df-41d7-b5c2-306971ed544c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:17.489390 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbdb948-f9df-41d7-b5c2-306971ed544c None admin] RBAC: Adding query filter params (role.id=2989d29f392d4d8c802354159891eb11, scope.project.id=71ad19503c5843ceb4595ebfa1cfb8dd, user.id=ebc0fa02ab024d6d86c055001e403304) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:54:17.489694 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbdb948-f9df-41d7-b5c2-306971ed544c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:17.491114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbdb948-f9df-41d7-b5c2-306971ed544c None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:17.496741 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 428/856] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Oct 23 19:54:17 2022] GET /identity/v3/role_assignments?user.id=ebc0fa02ab024d6d86c055001e403304&scope.project.id=71ad19503c5843ceb4595ebfa1cfb8dd&role.id=2989d29f392d4d8c802354159891eb11 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:19.614889 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92bdb9b-b3c4-4069-ae7f-6b4cc183164a None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:19.615436 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92bdb9b-b3c4-4069-ae7f-6b4cc183164a None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:19.615436 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92bdb9b-b3c4-4069-ae7f-6b4cc183164a None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:19.616764 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 429/857] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:54:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:54:19.622154 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b206d7e-81dc-452d-b878-ef026f60f764 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:19.622626 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b206d7e-81dc-452d-b878-ef026f60f764 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:19.622719 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b206d7e-81dc-452d-b878-ef026f60f764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:19.640861 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-7b206d7e-81dc-452d-b878-ef026f60f764 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:54:19.642038 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7b206d7e-81dc-452d-b878-ef026f60f764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:54:19.706853 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 429/858] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:54:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:54:19.973623 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92bdb9b-b3c4-4069-ae7f-6b4cc183164a None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:19.986294 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:20.073645 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92bdb9b-b3c4-4069-ae7f-6b4cc183164a None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:20.106472 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba50ab-7679-4372-8bad-70486e5155d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:20.107093 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba50ab-7679-4372-8bad-70486e5155d5 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:20.107373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba50ab-7679-4372-8bad-70486e5155d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:20.107646 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba50ab-7679-4372-8bad-70486e5155d5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:20.109327 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba50ab-7679-4372-8bad-70486e5155d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:20.110795 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba50ab-7679-4372-8bad-70486e5155d5 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:20.112374 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 430/859] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:54:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:20.118161 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b206d7e-81dc-452d-b878-ef026f60f764 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:20.173488 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:20.212775 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b206d7e-81dc-452d-b878-ef026f60f764 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:20.245700 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7715e567-55e3-4912-a829-9ff1d1070a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:20.246285 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715e567-55e3-4912-a829-9ff1d1070a6b None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:20.246616 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715e567-55e3-4912-a829-9ff1d1070a6b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:20.246894 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7715e567-55e3-4912-a829-9ff1d1070a6b None admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:20.247537 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7715e567-55e3-4912-a829-9ff1d1070a6b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:20.249003 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7715e567-55e3-4912-a829-9ff1d1070a6b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:20.275140 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 430/860] 10.222.0.13 () {60 vars in 1141 bytes} [Sun Oct 23 19:54:20 2022] POST /identity/v3/projects => generated 311 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:54:22.403123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb414462-74a1-4be1-8044-c3af2b42cdf4 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:22.403645 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb414462-74a1-4be1-8044-c3af2b42cdf4 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:22.403763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb414462-74a1-4be1-8044-c3af2b42cdf4 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:22.405015 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 431/861] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:54:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:54:22.410191 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b9497-afa5-4158-a368-78d1de37dc2d None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:22.410586 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b9497-afa5-4158-a368-78d1de37dc2d None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:22.410770 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b9497-afa5-4158-a368-78d1de37dc2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:22.428412 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-c82b9497-afa5-4158-a368-78d1de37dc2d None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:54:22.429566 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c82b9497-afa5-4158-a368-78d1de37dc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:54:22.493172 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 431/862] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:54:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:54:22.759089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb414462-74a1-4be1-8044-c3af2b42cdf4 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:22.835442 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:22.857023 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb414462-74a1-4be1-8044-c3af2b42cdf4 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:22.890746 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c98da4d-e677-43d5-a12a-2c26a4e42794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:22.891363 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98da4d-e677-43d5-a12a-2c26a4e42794 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:22.891640 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98da4d-e677-43d5-a12a-2c26a4e42794 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:22.891913 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c98da4d-e677-43d5-a12a-2c26a4e42794 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:22.893012 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c98da4d-e677-43d5-a12a-2c26a4e42794 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:22.894459 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c98da4d-e677-43d5-a12a-2c26a4e42794 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:22.896008 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 432/863] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:54:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:22.901724 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b9497-afa5-4158-a368-78d1de37dc2d None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:22.941945 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:22.997892 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b9497-afa5-4158-a368-78d1de37dc2d None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:23.031707 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad7e806-b854-4cc9-b1f8-46f33dd29f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:23.032344 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7e806-b854-4cc9-b1f8-46f33dd29f8b None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:23.032624 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7e806-b854-4cc9-b1f8-46f33dd29f8b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:23.032895 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad7e806-b854-4cc9-b1f8-46f33dd29f8b None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:23.033544 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad7e806-b854-4cc9-b1f8-46f33dd29f8b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:23.035033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad7e806-b854-4cc9-b1f8-46f33dd29f8b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:23.056275 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 432/864] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Oct 23 19:54:22 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:54:25.185519 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4961f01-6131-4b0d-bcf7-6c10db62cf02 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:25.186629 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4961f01-6131-4b0d-bcf7-6c10db62cf02 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:25.186629 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4961f01-6131-4b0d-bcf7-6c10db62cf02 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:25.187970 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 433/865] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:54:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:54:25.193493 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d1014-c7ed-405e-bd0c-3625ba4cfdd8 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:25.193913 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d1014-c7ed-405e-bd0c-3625ba4cfdd8 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:25.194018 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d1014-c7ed-405e-bd0c-3625ba4cfdd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:25.212231 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-983d1014-c7ed-405e-bd0c-3625ba4cfdd8 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:54:25.213403 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-983d1014-c7ed-405e-bd0c-3625ba4cfdd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:54:25.277089 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 433/866] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:54:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:54:25.544252 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4961f01-6131-4b0d-bcf7-6c10db62cf02 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:25.758520 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4961f01-6131-4b0d-bcf7-6c10db62cf02 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:25.790449 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce197d8-32c2-425b-ab00-be23cfafb552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:25.791065 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce197d8-32c2-425b-ab00-be23cfafb552 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:25.791344 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce197d8-32c2-425b-ab00-be23cfafb552 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:25.791613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce197d8-32c2-425b-ab00-be23cfafb552 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:25.801959 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce197d8-32c2-425b-ab00-be23cfafb552 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:54:25.807003 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-2ce197d8-32c2-425b-ab00-be23cfafb552 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:54:25.807003 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:54:25.807003 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:54:25.807003 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:54:25.807003 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:54:25.807003 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:54:25.807003 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:54:25.807003 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:54:25.807003 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:54:25.807003 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:54:25.807003 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:54:25.807003 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:54:25.807003 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:54:25.807003 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:54:25.807003 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:54:25.807003 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:54:25.808123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:54:25.808123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:54:25.808123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:54:25.808123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:54:25.808123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:54:25.808123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:54:25.808123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:54:25.808123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:54:25.808123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:54:25.808123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:54:25.808123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:54:25.808123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:54:25.808123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:54:25.808123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:54:25.808123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:54:25.808123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:54:25.809369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:54:25.809369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:54:25.809369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:54:25.809369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:54:25.809369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:54:25.809369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:54:25.809369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:54:25.809369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:54:25.809369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:54:25.809369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:54:25.809369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:54:25.809369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:54:25.809369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 434/867] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Oct 23 19:54:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:54:25.814981 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d1014-c7ed-405e-bd0c-3625ba4cfdd8 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:25.840440 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:25.910295 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d1014-c7ed-405e-bd0c-3625ba4cfdd8 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:25.942830 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5daaca-59d6-4016-9c60-74ad10c58487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:25.943417 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5daaca-59d6-4016-9c60-74ad10c58487 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:25.943692 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5daaca-59d6-4016-9c60-74ad10c58487 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:25.943963 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5daaca-59d6-4016-9c60-74ad10c58487 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:25.944622 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5daaca-59d6-4016-9c60-74ad10c58487 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:54:25.944923 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5daaca-59d6-4016-9c60-74ad10c58487 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:25.946283 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5daaca-59d6-4016-9c60-74ad10c58487 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:25.953941 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 434/868] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:54:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:25.959540 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce197d8-32c2-425b-ab00-be23cfafb552 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:25.968639 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:26.058922 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce197d8-32c2-425b-ab00-be23cfafb552 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:26.091367 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4a84a-d5cd-4c1b-849d-a1ebe197d98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:26.091990 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4a84a-d5cd-4c1b-849d-a1ebe197d98d None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:26.092767 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4a84a-d5cd-4c1b-849d-a1ebe197d98d None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:26.093056 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a4a84a-d5cd-4c1b-849d-a1ebe197d98d None admin] PATH_INFO: `/v3/users/66efd1b52c8f4bd7bea1fdb1214b2370` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:26.105591 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a4a84a-d5cd-4c1b-849d-a1ebe197d98d None admin] RBAC: Authorizing `identity:get_user(user_id=66efd1b52c8f4bd7bea1fdb1214b2370)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:26.107062 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a4a84a-d5cd-4c1b-849d-a1ebe197d98d None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:26.109201 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 435/869] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:54:25 2022] GET /identity/v3/users/66efd1b52c8f4bd7bea1fdb1214b2370 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:26.114875 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5daaca-59d6-4016-9c60-74ad10c58487 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:26.199740 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:26.211880 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5daaca-59d6-4016-9c60-74ad10c58487 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:26.244728 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d2b02-192a-4f3c-a782-c988efb8c8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:26.245355 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d2b02-192a-4f3c-a782-c988efb8c8d1 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:26.245632 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d2b02-192a-4f3c-a782-c988efb8c8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:26.245904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d2b02-192a-4f3c-a782-c988efb8c8d1 None admin] PATH_INFO: `/v3/projects/087311d3a56c486d880f4e0086dd82d6` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:26.247033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244d2b02-192a-4f3c-a782-c988efb8c8d1 None admin] RBAC: Authorizing `identity:get_project(project_id=087311d3a56c486d880f4e0086dd82d6)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:26.248446 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244d2b02-192a-4f3c-a782-c988efb8c8d1 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:26.250045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 435/870] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Oct 23 19:54:26 2022] GET /identity/v3/projects/087311d3a56c486d880f4e0086dd82d6 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:26.255915 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4a84a-d5cd-4c1b-849d-a1ebe197d98d None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:26.315879 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:26.352582 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a4a84a-d5cd-4c1b-849d-a1ebe197d98d None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:26.384581 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbad572-7c55-4235-999e-45fd7d8144ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:26.385220 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbad572-7c55-4235-999e-45fd7d8144ea None admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:26.385497 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbad572-7c55-4235-999e-45fd7d8144ea None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:26.385767 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbad572-7c55-4235-999e-45fd7d8144ea None admin] PATH_INFO: `/v3/projects/087311d3a56c486d880f4e0086dd82d6/users/66efd1b52c8f4bd7bea1fdb1214b2370/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:26.387663 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbad572-7c55-4235-999e-45fd7d8144ea None admin] RBAC: Authorizing `identity:create_grant(project_id=087311d3a56c486d880f4e0086dd82d6, user_id=66efd1b52c8f4bd7bea1fdb1214b2370, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:26.389586 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbad572-7c55-4235-999e-45fd7d8144ea None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:26.401897 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 436/871] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Oct 23 19:54:26 2022] PUT /identity/v3/projects/087311d3a56c486d880f4e0086dd82d6/users/66efd1b52c8f4bd7bea1fdb1214b2370/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 19:54:28.497594 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903a984-5ddb-49cd-8031-60d38e3f5298 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:28.498151 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903a984-5ddb-49cd-8031-60d38e3f5298 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:28.498151 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b903a984-5ddb-49cd-8031-60d38e3f5298 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:28.500014 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 436/872] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:54:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:54:28.505193 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df03221-7736-457d-a8ff-3613c17e8bfe None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:28.505611 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df03221-7736-457d-a8ff-3613c17e8bfe None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:28.505743 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df03221-7736-457d-a8ff-3613c17e8bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:28.524022 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-1df03221-7736-457d-a8ff-3613c17e8bfe None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:54:28.525192 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-1df03221-7736-457d-a8ff-3613c17e8bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:54:28.589432 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 437/873] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:54:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:54:28.856971 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903a984-5ddb-49cd-8031-60d38e3f5298 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:28.954974 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b903a984-5ddb-49cd-8031-60d38e3f5298 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:28.987093 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5692da4f-063c-433c-927a-3c74e4d88aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:28.987696 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692da4f-063c-433c-927a-3c74e4d88aea None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:28.987969 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692da4f-063c-433c-927a-3c74e4d88aea None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:28.988237 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692da4f-063c-433c-927a-3c74e4d88aea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:28.997784 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5692da4f-063c-433c-927a-3c74e4d88aea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:54:29.003060 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-5692da4f-063c-433c-927a-3c74e4d88aea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:54:29.003060 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:54:29.003060 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:54:29.003060 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:54:29.003060 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:54:29.003060 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:54:29.003060 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:54:29.003060 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:54:29.003060 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:54:29.003060 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:54:29.003060 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:54:29.003060 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:54:29.003060 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:54:29.003060 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:54:29.003060 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:54:29.003060 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:54:29.004136 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:54:29.004136 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:54:29.004136 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:54:29.004136 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:54:29.004136 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:54:29.004136 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:54:29.004136 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:54:29.004136 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:54:29.004136 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:54:29.004136 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:54:29.004136 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:54:29.004136 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:54:29.004136 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:54:29.004136 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:54:29.004136 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:54:29.004136 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:54:29.005292 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:54:29.005292 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:54:29.005292 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:54:29.005292 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:54:29.005292 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:54:29.005292 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:54:29.005292 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:54:29.005292 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:54:29.005292 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:54:29.005292 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:54:29.005292 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:54:29.005292 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:54:29.005292 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 437/874] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Oct 23 19:54:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:54:29.009912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df03221-7736-457d-a8ff-3613c17e8bfe None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:29.063879 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:29.107748 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df03221-7736-457d-a8ff-3613c17e8bfe None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:29.140943 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c1edd-2913-4ae3-89c7-7b15e1961be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:29.141539 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c1edd-2913-4ae3-89c7-7b15e1961be3 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:29.141817 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c1edd-2913-4ae3-89c7-7b15e1961be3 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:29.142088 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c1edd-2913-4ae3-89c7-7b15e1961be3 None admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:29.142770 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c1edd-2913-4ae3-89c7-7b15e1961be3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:54:29.143070 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c1edd-2913-4ae3-89c7-7b15e1961be3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:29.144414 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c1edd-2913-4ae3-89c7-7b15e1961be3 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:29.152062 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 438/875] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:54:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:29.158739 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5692da4f-063c-433c-927a-3c74e4d88aea None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:29.176121 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:29.254570 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5692da4f-063c-433c-927a-3c74e4d88aea None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:29.287249 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d8c2e-ecda-4146-9577-1a7679c7bd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:29.287845 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d8c2e-ecda-4146-9577-1a7679c7bd7b None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:29.288118 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d8c2e-ecda-4146-9577-1a7679c7bd7b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:29.288386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9d8c2e-ecda-4146-9577-1a7679c7bd7b None admin] PATH_INFO: `/v3/users/66efd1b52c8f4bd7bea1fdb1214b2370` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:29.289466 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d8c2e-ecda-4146-9577-1a7679c7bd7b None admin] RBAC: Authorizing `identity:get_user(user_id=66efd1b52c8f4bd7bea1fdb1214b2370)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:29.290889 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9d8c2e-ecda-4146-9577-1a7679c7bd7b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:29.292425 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 438/876] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:54:29 2022] GET /identity/v3/users/66efd1b52c8f4bd7bea1fdb1214b2370 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:29.298750 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c1edd-2913-4ae3-89c7-7b15e1961be3 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:29.394481 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c1edd-2913-4ae3-89c7-7b15e1961be3 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:29.409512 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:29.427354 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7a1a18-e04b-4013-9019-ae37a0ba88d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:29.427943 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a1a18-e04b-4013-9019-ae37a0ba88d2 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:29.428221 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a1a18-e04b-4013-9019-ae37a0ba88d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:29.428489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7a1a18-e04b-4013-9019-ae37a0ba88d2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:29.492367 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7a1a18-e04b-4013-9019-ae37a0ba88d2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:29.493782 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7a1a18-e04b-4013-9019-ae37a0ba88d2 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:29.587372 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 439/877] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Oct 23 19:54:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 23 19:54:29.593884 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d8c2e-ecda-4146-9577-1a7679c7bd7b None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:29.674671 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:29.691328 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9d8c2e-ecda-4146-9577-1a7679c7bd7b None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:29.723829 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06678c08-143d-40f5-860e-9b12c840d976 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:29.724449 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06678c08-143d-40f5-860e-9b12c840d976 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:29.724728 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06678c08-143d-40f5-860e-9b12c840d976 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:29.725000 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06678c08-143d-40f5-860e-9b12c840d976 None admin] PATH_INFO: `/v3/projects/087311d3a56c486d880f4e0086dd82d6` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:29.726092 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06678c08-143d-40f5-860e-9b12c840d976 None admin] RBAC: Authorizing `identity:get_project(project_id=087311d3a56c486d880f4e0086dd82d6)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:29.727522 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06678c08-143d-40f5-860e-9b12c840d976 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:29.729130 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 439/878] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Oct 23 19:54:29 2022] GET /identity/v3/projects/087311d3a56c486d880f4e0086dd82d6 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:29.735174 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7a1a18-e04b-4013-9019-ae37a0ba88d2 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:29.752502 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:29.831477 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7a1a18-e04b-4013-9019-ae37a0ba88d2 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:29.863827 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437437d-0909-4f23-81b9-a2345cc71cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:29.864411 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437437d-0909-4f23-81b9-a2345cc71cba None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:29.864683 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437437d-0909-4f23-81b9-a2345cc71cba None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:29.864948 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437437d-0909-4f23-81b9-a2345cc71cba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:29.865663 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5437437d-0909-4f23-81b9-a2345cc71cba None admin] RBAC: Adding query filter params (role.id=742dfaf63ef74207a2c9213411513925, scope.project.id=087311d3a56c486d880f4e0086dd82d6, user.id=66efd1b52c8f4bd7bea1fdb1214b2370) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:54:29.865968 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5437437d-0909-4f23-81b9-a2345cc71cba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:29.867399 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5437437d-0909-4f23-81b9-a2345cc71cba None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:29.872940 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 440/879] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Oct 23 19:54:29 2022] GET /identity/v3/role_assignments?user.id=66efd1b52c8f4bd7bea1fdb1214b2370&scope.project.id=087311d3a56c486d880f4e0086dd82d6&role.id=742dfaf63ef74207a2c9213411513925 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:32.005119 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0fd852-0dd5-4dc1-955f-b0410564f5a1 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:32.005698 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0fd852-0dd5-4dc1-955f-b0410564f5a1 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:32.005698 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0fd852-0dd5-4dc1-955f-b0410564f5a1 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:32.007197 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 440/880] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:54:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:54:32.012324 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e162a5f-4e25-44c9-ae73-30f368a814fe None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:32.012741 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e162a5f-4e25-44c9-ae73-30f368a814fe None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:32.012854 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e162a5f-4e25-44c9-ae73-30f368a814fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:32.031350 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-3e162a5f-4e25-44c9-ae73-30f368a814fe None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:54:32.032532 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-3e162a5f-4e25-44c9-ae73-30f368a814fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:54:32.096180 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 441/881] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:54:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:54:32.363739 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0fd852-0dd5-4dc1-955f-b0410564f5a1 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:32.462828 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0fd852-0dd5-4dc1-955f-b0410564f5a1 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:32.495086 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-119e210e-9c56-48c1-8a39-84bed869266d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:32.495677 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e210e-9c56-48c1-8a39-84bed869266d None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:32.495954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e210e-9c56-48c1-8a39-84bed869266d None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:32.496226 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-119e210e-9c56-48c1-8a39-84bed869266d None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:32.496876 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119e210e-9c56-48c1-8a39-84bed869266d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:32.498212 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-119e210e-9c56-48c1-8a39-84bed869266d None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:32.513602 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 441/882] 10.222.0.13 () {60 vars in 1138 bytes} [Sun Oct 23 19:54:32 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:54:34.631328 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb96a7-3223-4113-8da0-6c31ef9a5be3 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:34.631903 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb96a7-3223-4113-8da0-6c31ef9a5be3 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:34.631903 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ceb96a7-3223-4113-8da0-6c31ef9a5be3 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:34.633205 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 442/883] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:54:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:54:34.638534 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8c322-f37a-49f2-82f3-c1c7590df6e0 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:34.639471 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8c322-f37a-49f2-82f3-c1c7590df6e0 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:34.639576 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8c322-f37a-49f2-82f3-c1c7590df6e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:34.657796 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-15b8c322-f37a-49f2-82f3-c1c7590df6e0 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:54:34.659008 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-15b8c322-f37a-49f2-82f3-c1c7590df6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:54:34.666077 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:34.724277 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 442/884] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:54:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:54:34.994042 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb96a7-3223-4113-8da0-6c31ef9a5be3 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:35.006699 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:35.102502 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ceb96a7-3223-4113-8da0-6c31ef9a5be3 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:35.135975 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798f41a-641b-4e7c-a93b-3dc306e8a7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:35.136591 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798f41a-641b-4e7c-a93b-3dc306e8a7b4 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:35.136864 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798f41a-641b-4e7c-a93b-3dc306e8a7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:35.137129 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798f41a-641b-4e7c-a93b-3dc306e8a7b4 None admin] PATH_INFO: `/v3/domains/09c1d12b65b54035811d9f699a0c584f` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:35.147809 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7798f41a-641b-4e7c-a93b-3dc306e8a7b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=09c1d12b65b54035811d9f699a0c584f)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:35.149327 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7798f41a-641b-4e7c-a93b-3dc306e8a7b4 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:35.150963 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 443/885] 10.222.0.13 () {56 vars in 1183 bytes} [Sun Oct 23 19:54:34 2022] GET /identity/v3/domains/09c1d12b65b54035811d9f699a0c584f => generated 267 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:35.156914 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8c322-f37a-49f2-82f3-c1c7590df6e0 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:35.252950 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8c322-f37a-49f2-82f3-c1c7590df6e0 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:35.285764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5384c7c-2d07-44ab-95fe-bbadb7813129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:35.286408 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5384c7c-2d07-44ab-95fe-bbadb7813129 None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:35.286674 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5384c7c-2d07-44ab-95fe-bbadb7813129 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:35.286947 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5384c7c-2d07-44ab-95fe-bbadb7813129 None admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:35.287610 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5384c7c-2d07-44ab-95fe-bbadb7813129 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:35.289029 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5384c7c-2d07-44ab-95fe-bbadb7813129 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:35.303444 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:35.317348 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 443/886] 10.222.0.13 () {60 vars in 1141 bytes} [Sun Oct 23 19:54:35 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:54:37.477713 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc34c3f-d9cd-4efa-bb30-d23d729c21d1 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:37.478279 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc34c3f-d9cd-4efa-bb30-d23d729c21d1 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:37.478279 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc34c3f-d9cd-4efa-bb30-d23d729c21d1 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:37.479624 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 444/887] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:54:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:54:37.484801 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc354e9b-633b-404b-aebf-f4ffa26a3315 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:37.485233 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc354e9b-633b-404b-aebf-f4ffa26a3315 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:37.485355 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc354e9b-633b-404b-aebf-f4ffa26a3315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:37.503471 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-fc354e9b-633b-404b-aebf-f4ffa26a3315 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:54:37.504636 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-fc354e9b-633b-404b-aebf-f4ffa26a3315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:54:37.568605 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 444/888] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:54:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:54:37.838733 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc34c3f-d9cd-4efa-bb30-d23d729c21d1 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:37.907388 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:37.938950 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc34c3f-d9cd-4efa-bb30-d23d729c21d1 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:37.971255 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc4de4-4d08-4445-8752-d5eba427f672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:37.971869 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc4de4-4d08-4445-8752-d5eba427f672 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:37.972144 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc4de4-4d08-4445-8752-d5eba427f672 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:37.972416 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc4de4-4d08-4445-8752-d5eba427f672 None admin] PATH_INFO: `/v3/domains/09c1d12b65b54035811d9f699a0c584f` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:37.973518 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fc4de4-4d08-4445-8752-d5eba427f672 None admin] RBAC: Authorizing `identity:get_domain(domain_id=09c1d12b65b54035811d9f699a0c584f)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:37.975068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fc4de4-4d08-4445-8752-d5eba427f672 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:37.976623 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 445/889] 10.222.0.13 () {56 vars in 1183 bytes} [Sun Oct 23 19:54:37 2022] GET /identity/v3/domains/09c1d12b65b54035811d9f699a0c584f => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:37.982594 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc354e9b-633b-404b-aebf-f4ffa26a3315 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:38.078256 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc354e9b-633b-404b-aebf-f4ffa26a3315 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:38.101152 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:38.112629 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad9d6e9-c75a-47b0-8b76-b2a74c4ca207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:38.113224 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9d6e9-c75a-47b0-8b76-b2a74c4ca207 None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:38.113498 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9d6e9-c75a-47b0-8b76-b2a74c4ca207 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:38.113763 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad9d6e9-c75a-47b0-8b76-b2a74c4ca207 None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:38.114465 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad9d6e9-c75a-47b0-8b76-b2a74c4ca207 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:38.115867 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad9d6e9-c75a-47b0-8b76-b2a74c4ca207 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:38.135386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 445/890] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Oct 23 19:54:37 2022] POST /identity/v3/users => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 19:54:40.255493 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e749536-80c6-4d9b-84bc-214cb099806b None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:40.256025 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e749536-80c6-4d9b-84bc-214cb099806b None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:40.256025 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e749536-80c6-4d9b-84bc-214cb099806b None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:40.257397 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 446/891] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:54:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:54:40.262449 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23c223-258e-4c60-bcef-4790bb150f05 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:40.262881 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23c223-258e-4c60-bcef-4790bb150f05 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:40.262974 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23c223-258e-4c60-bcef-4790bb150f05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:40.281212 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-6b23c223-258e-4c60-bcef-4790bb150f05 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:54:40.282354 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-6b23c223-258e-4c60-bcef-4790bb150f05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:54:40.346458 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 446/892] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:54:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:54:40.610190 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e749536-80c6-4d9b-84bc-214cb099806b None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:40.707386 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e749536-80c6-4d9b-84bc-214cb099806b None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:40.739900 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7c248-ec4b-4786-b26b-92a1078fb5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:40.740505 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7c248-ec4b-4786-b26b-92a1078fb5c9 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:40.740780 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7c248-ec4b-4786-b26b-92a1078fb5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:40.741045 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d7c248-ec4b-4786-b26b-92a1078fb5c9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:40.747484 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:40.752705 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d7c248-ec4b-4786-b26b-92a1078fb5c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:54:40.757444 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-e5d7c248-ec4b-4786-b26b-92a1078fb5c9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:54:40.757444 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:54:40.757444 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:54:40.757444 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:54:40.757444 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:54:40.757444 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:54:40.757444 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:54:40.757444 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:54:40.757444 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:54:40.757444 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:54:40.757444 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:54:40.757444 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:54:40.757444 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:54:40.757444 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:54:40.757444 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:54:40.757444 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:54:40.758615 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:54:40.758615 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:54:40.758615 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:54:40.758615 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:54:40.758615 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:54:40.758615 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:54:40.758615 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:54:40.758615 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:54:40.758615 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:54:40.758615 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:54:40.758615 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:54:40.758615 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:54:40.758615 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:54:40.758615 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:54:40.758615 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:54:40.758615 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:54:40.759521 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:54:40.759521 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:54:40.759521 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:54:40.759521 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:54:40.759521 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:54:40.759521 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:54:40.759521 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:54:40.759521 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:54:40.759521 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:54:40.759521 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:54:40.759521 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:54:40.759521 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:54:40.759521 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 447/893] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Oct 23 19:54:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:54:40.764547 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23c223-258e-4c60-bcef-4790bb150f05 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:40.860770 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23c223-258e-4c60-bcef-4790bb150f05 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:40.870965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:40.894877 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb4016-7f44-42f1-ba5e-ef649db27ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:40.895472 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb4016-7f44-42f1-ba5e-ef649db27ed2 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:40.895747 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb4016-7f44-42f1-ba5e-ef649db27ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:40.896017 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fb4016-7f44-42f1-ba5e-ef649db27ed2 None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:40.896668 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb4016-7f44-42f1-ba5e-ef649db27ed2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:54:40.896965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb4016-7f44-42f1-ba5e-ef649db27ed2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:40.898347 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fb4016-7f44-42f1-ba5e-ef649db27ed2 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:40.906537 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 447/894] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:54:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:54:40.912413 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7c248-ec4b-4786-b26b-92a1078fb5c9 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:41.007720 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d7c248-ec4b-4786-b26b-92a1078fb5c9 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:41.040156 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e6d98-4ee4-42ad-91da-998f4ddf99af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:41.040778 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e6d98-4ee4-42ad-91da-998f4ddf99af None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:41.041056 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e6d98-4ee4-42ad-91da-998f4ddf99af None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:41.041328 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e6d98-4ee4-42ad-91da-998f4ddf99af None admin] PATH_INFO: `/v3/users/0c8d43f6ceb14720ba8a4eeb201cc92c` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:41.054512 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e6d98-4ee4-42ad-91da-998f4ddf99af None admin] RBAC: Authorizing `identity:get_user(user_id=0c8d43f6ceb14720ba8a4eeb201cc92c)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:41.055927 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e6d98-4ee4-42ad-91da-998f4ddf99af None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:41.057510 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 448/895] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:54:40 2022] GET /identity/v3/users/0c8d43f6ceb14720ba8a4eeb201cc92c => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:41.063235 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb4016-7f44-42f1-ba5e-ef649db27ed2 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:41.159374 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fb4016-7f44-42f1-ba5e-ef649db27ed2 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:41.191865 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a3d1c-414d-46d7-bb38-ff1d7e422050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:41.192456 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a3d1c-414d-46d7-bb38-ff1d7e422050 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:41.192732 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a3d1c-414d-46d7-bb38-ff1d7e422050 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:41.193002 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a3d1c-414d-46d7-bb38-ff1d7e422050 None admin] PATH_INFO: `/v3/projects/bf6c5162b1e04c2c8eb3f5bfe87acc2b` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:41.194089 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a3d1c-414d-46d7-bb38-ff1d7e422050 None admin] RBAC: Authorizing `identity:get_project(project_id=bf6c5162b1e04c2c8eb3f5bfe87acc2b)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:41.195514 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393a3d1c-414d-46d7-bb38-ff1d7e422050 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:41.197100 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 448/896] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Oct 23 19:54:41 2022] GET /identity/v3/projects/bf6c5162b1e04c2c8eb3f5bfe87acc2b => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:41.202719 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e6d98-4ee4-42ad-91da-998f4ddf99af None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:41.256665 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:41.299261 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e6d98-4ee4-42ad-91da-998f4ddf99af None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:41.331683 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca00808-9aed-4495-a604-30e917548b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:41.332322 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca00808-9aed-4495-a604-30e917548b76 None admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:41.332597 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca00808-9aed-4495-a604-30e917548b76 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:41.332865 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca00808-9aed-4495-a604-30e917548b76 None admin] PATH_INFO: `/v3/projects/bf6c5162b1e04c2c8eb3f5bfe87acc2b/users/0c8d43f6ceb14720ba8a4eeb201cc92c/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:41.334734 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca00808-9aed-4495-a604-30e917548b76 None admin] RBAC: Authorizing `identity:create_grant(project_id=bf6c5162b1e04c2c8eb3f5bfe87acc2b, user_id=0c8d43f6ceb14720ba8a4eeb201cc92c, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:41.336133 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca00808-9aed-4495-a604-30e917548b76 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:41.348543 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 449/897] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Oct 23 19:54:41 2022] PUT /identity/v3/projects/bf6c5162b1e04c2c8eb3f5bfe87acc2b/users/0c8d43f6ceb14720ba8a4eeb201cc92c/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 19:54:43.443988 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6d994-0e7d-45ea-95e6-8500b78dd154 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:43.444510 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6d994-0e7d-45ea-95e6-8500b78dd154 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:43.444510 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd6d994-0e7d-45ea-95e6-8500b78dd154 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:43.445859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 449/898] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:54:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:54:43.451090 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0a24e-f454-48a8-a48f-d3e944252853 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:43.452071 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0a24e-f454-48a8-a48f-d3e944252853 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:43.452225 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc0a24e-f454-48a8-a48f-d3e944252853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:43.469798 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-cbc0a24e-f454-48a8-a48f-d3e944252853 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:54:43.471058 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-cbc0a24e-f454-48a8-a48f-d3e944252853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:54:43.534722 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 450/899] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:54:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:54:43.799290 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6d994-0e7d-45ea-95e6-8500b78dd154 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:43.841091 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:43.897989 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd6d994-0e7d-45ea-95e6-8500b78dd154 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:43.930767 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3671dcff-73c6-424f-8231-25eb75362298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:43.931376 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3671dcff-73c6-424f-8231-25eb75362298 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:43.931652 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3671dcff-73c6-424f-8231-25eb75362298 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:43.931923 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3671dcff-73c6-424f-8231-25eb75362298 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:43.941908 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3671dcff-73c6-424f-8231-25eb75362298 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:54:43.946673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-3671dcff-73c6-424f-8231-25eb75362298 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:54:43.946673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:54:43.946673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:54:43.946673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:54:43.946673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:54:43.946673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:54:43.946673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:54:43.946673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:54:43.946673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:54:43.946673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:54:43.946673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:54:43.946673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:54:43.946673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:54:43.946673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:54:43.946673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:54:43.946673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:54:43.947753 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:54:43.947753 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:54:43.947753 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:54:43.947753 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:54:43.947753 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:54:43.947753 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:54:43.947753 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:54:43.947753 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:54:43.947753 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:54:43.947753 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:54:43.947753 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:54:43.947753 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:54:43.947753 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:54:43.947753 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:54:43.947753 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:54:43.947753 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:54:43.948778 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:54:43.948778 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:54:43.948778 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:54:43.948778 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:54:43.948778 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:54:43.948778 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:54:43.948778 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:54:43.948778 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:54:43.948778 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:54:43.948778 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:54:43.948778 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:54:43.948778 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:54:43.948778 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 450/900] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Oct 23 19:54:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:54:43.953745 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0a24e-f454-48a8-a48f-d3e944252853 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:43.997809 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:44.050639 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc0a24e-f454-48a8-a48f-d3e944252853 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:44.082710 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf7fdc-104b-4b22-866c-455fb745514c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:44.083301 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf7fdc-104b-4b22-866c-455fb745514c None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:44.083574 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf7fdc-104b-4b22-866c-455fb745514c None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:44.083842 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebf7fdc-104b-4b22-866c-455fb745514c None admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:44.084519 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebf7fdc-104b-4b22-866c-455fb745514c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:54:44.084816 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebf7fdc-104b-4b22-866c-455fb745514c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:44.086162 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebf7fdc-104b-4b22-866c-455fb745514c None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:44.094405 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 451/901] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:54:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:44.100740 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3671dcff-73c6-424f-8231-25eb75362298 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:44.193319 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:44.196284 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3671dcff-73c6-424f-8231-25eb75362298 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:44.229694 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe9b5f-6bce-4d5f-a90e-e339cee008b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:44.230325 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe9b5f-6bce-4d5f-a90e-e339cee008b3 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:44.230601 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe9b5f-6bce-4d5f-a90e-e339cee008b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:44.230869 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffe9b5f-6bce-4d5f-a90e-e339cee008b3 None admin] PATH_INFO: `/v3/users/0c8d43f6ceb14720ba8a4eeb201cc92c` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:44.231951 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffe9b5f-6bce-4d5f-a90e-e339cee008b3 None admin] RBAC: Authorizing `identity:get_user(user_id=0c8d43f6ceb14720ba8a4eeb201cc92c)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:44.233987 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffe9b5f-6bce-4d5f-a90e-e339cee008b3 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:44.235566 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 451/902] 10.222.0.13 () {56 vars in 1177 bytes} [Sun Oct 23 19:54:44 2022] GET /identity/v3/users/0c8d43f6ceb14720ba8a4eeb201cc92c => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:44.241148 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf7fdc-104b-4b22-866c-455fb745514c None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:44.336129 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebf7fdc-104b-4b22-866c-455fb745514c None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:44.345032 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:44.368632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c63691-40b3-4ac9-a109-1c79fa955d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:44.369226 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c63691-40b3-4ac9-a109-1c79fa955d27 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:44.369506 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c63691-40b3-4ac9-a109-1c79fa955d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:44.369773 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c63691-40b3-4ac9-a109-1c79fa955d27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:44.433010 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c63691-40b3-4ac9-a109-1c79fa955d27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:44.434455 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c63691-40b3-4ac9-a109-1c79fa955d27 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:44.528478 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 452/903] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Oct 23 19:54:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 23 19:54:44.534882 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe9b5f-6bce-4d5f-a90e-e339cee008b3 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:44.628960 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffe9b5f-6bce-4d5f-a90e-e339cee008b3 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:44.661557 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ba1ca9-6f28-45e5-b45c-ce0d64be911e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:44.662165 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba1ca9-6f28-45e5-b45c-ce0d64be911e None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:44.662467 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba1ca9-6f28-45e5-b45c-ce0d64be911e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:44.662738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ba1ca9-6f28-45e5-b45c-ce0d64be911e None admin] PATH_INFO: `/v3/projects/bf6c5162b1e04c2c8eb3f5bfe87acc2b` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:44.663825 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba1ca9-6f28-45e5-b45c-ce0d64be911e None admin] RBAC: Authorizing `identity:get_project(project_id=bf6c5162b1e04c2c8eb3f5bfe87acc2b)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:44.665222 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ba1ca9-6f28-45e5-b45c-ce0d64be911e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:44.666826 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 452/904] 10.222.0.13 () {56 vars in 1186 bytes} [Sun Oct 23 19:54:44 2022] GET /identity/v3/projects/bf6c5162b1e04c2c8eb3f5bfe87acc2b => generated 361 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:44.672598 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c63691-40b3-4ac9-a109-1c79fa955d27 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:54:44.682356 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:44.767908 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c63691-40b3-4ac9-a109-1c79fa955d27 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:54:44.801166 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4731994-2909-4605-a277-f8ff3cab2f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:54:44.801760 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4731994-2909-4605-a277-f8ff3cab2f6a None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:54:44.802039 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4731994-2909-4605-a277-f8ff3cab2f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:54:44.802338 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4731994-2909-4605-a277-f8ff3cab2f6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:54:44.803062 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4731994-2909-4605-a277-f8ff3cab2f6a None admin] RBAC: Adding query filter params (role.id=742dfaf63ef74207a2c9213411513925, scope.project.id=bf6c5162b1e04c2c8eb3f5bfe87acc2b, user.id=0c8d43f6ceb14720ba8a4eeb201cc92c) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:54:44.803369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4731994-2909-4605-a277-f8ff3cab2f6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:54:44.804776 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4731994-2909-4605-a277-f8ff3cab2f6a None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:54:44.810402 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 453/905] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Oct 23 19:54:44 2022] GET /identity/v3/role_assignments?user.id=0c8d43f6ceb14720ba8a4eeb201cc92c&scope.project.id=bf6c5162b1e04c2c8eb3f5bfe87acc2b&role.id=742dfaf63ef74207a2c9213411513925 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:54:54.657595 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=82 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:54:54.807788 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=64 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:39.789671 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88178d-9ae6-4e25-a19d-6cefdec096d4 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:39.789671 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88178d-9ae6-4e25-a19d-6cefdec096d4 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:39.789671 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd88178d-9ae6-4e25-a19d-6cefdec096d4 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:39.790708 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 453/906] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:55:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:55:39.796540 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d7525-b9ad-456b-9fc7-90826ff3da0d None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:39.797022 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d7525-b9ad-456b-9fc7-90826ff3da0d None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:39.797239 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5d7525-b9ad-456b-9fc7-90826ff3da0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:39.817803 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-5a5d7525-b9ad-456b-9fc7-90826ff3da0d None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:55:39.819104 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-5a5d7525-b9ad-456b-9fc7-90826ff3da0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:55:39.914413 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 454/907] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:55:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:55:40.176243 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88178d-9ae6-4e25-a19d-6cefdec096d4 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:40.300985 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd88178d-9ae6-4e25-a19d-6cefdec096d4 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:40.342084 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb74af1-46e4-4f2d-ad05-6c7051ba6afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:40.343356 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb74af1-46e4-4f2d-ad05-6c7051ba6afc None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:40.343950 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb74af1-46e4-4f2d-ad05-6c7051ba6afc None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:40.344421 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb74af1-46e4-4f2d-ad05-6c7051ba6afc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:40.356788 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb74af1-46e4-4f2d-ad05-6c7051ba6afc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:40.360098 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb74af1-46e4-4f2d-ad05-6c7051ba6afc None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:40.378349 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-2eb74af1-46e4-4f2d-ad05-6c7051ba6afc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:55:40.378349 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:55:40.378349 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:55:40.378349 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:55:40.378349 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:55:40.378349 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:55:40.378349 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:55:40.378349 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:55:40.378349 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:55:40.378349 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:55:40.378349 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:55:40.378349 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:55:40.378349 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:55:40.378349 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:55:40.378349 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:55:40.378349 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:55:40.379646 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:55:40.379646 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:55:40.379646 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:55:40.379646 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:55:40.379646 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:55:40.379646 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:55:40.379646 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:55:40.379646 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:55:40.379646 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:55:40.379646 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:55:40.379646 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:55:40.379646 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:55:40.379646 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:55:40.379646 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:55:40.379646 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:55:40.379646 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:55:40.379646 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:55:40.381079 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:55:40.381079 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:55:40.381079 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:55:40.382733 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 454/908] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:55:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:55:40.389449 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d7525-b9ad-456b-9fc7-90826ff3da0d None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:40.509365 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5d7525-b9ad-456b-9fc7-90826ff3da0d None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:40.540309 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:40.548435 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426a7da-f037-42e4-8e5f-07c96897cb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:40.549122 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426a7da-f037-42e4-8e5f-07c96897cb99 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:40.549576 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426a7da-f037-42e4-8e5f-07c96897cb99 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:40.549933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426a7da-f037-42e4-8e5f-07c96897cb99 None admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:40.550635 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426a7da-f037-42e4-8e5f-07c96897cb99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:55:40.550941 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426a7da-f037-42e4-8e5f-07c96897cb99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:40.552312 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426a7da-f037-42e4-8e5f-07c96897cb99 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:40.573769 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 455/909] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:55:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:55:40.575791 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb74af1-46e4-4f2d-ad05-6c7051ba6afc None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:40.639259 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:40.692523 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb74af1-46e4-4f2d-ad05-6c7051ba6afc None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:40.731175 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6738bd24-a8b2-4dc8-a3d4-ff4a6317e034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:40.732409 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738bd24-a8b2-4dc8-a3d4-ff4a6317e034 None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:40.732825 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738bd24-a8b2-4dc8-a3d4-ff4a6317e034 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:40.733106 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738bd24-a8b2-4dc8-a3d4-ff4a6317e034 None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:40.733753 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6738bd24-a8b2-4dc8-a3d4-ff4a6317e034 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:40.735412 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6738bd24-a8b2-4dc8-a3d4-ff4a6317e034 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:40.770511 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 455/910] 10.222.0.13 () {60 vars in 1132 bytes} [Sun Oct 23 19:55:40 2022] POST /identity/v3/users => generated 251 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:55:43.102167 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29815b-c335-437b-be16-48c82b5da318 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:43.102742 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29815b-c335-437b-be16-48c82b5da318 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:43.102742 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29815b-c335-437b-be16-48c82b5da318 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:43.104109 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 456/911] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:55:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:55:43.110501 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22130a1-66c0-4a0c-b42e-1126795d81a0 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:43.110972 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22130a1-66c0-4a0c-b42e-1126795d81a0 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:43.111074 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22130a1-66c0-4a0c-b42e-1126795d81a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:43.134818 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-a22130a1-66c0-4a0c-b42e-1126795d81a0 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:55:43.136191 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a22130a1-66c0-4a0c-b42e-1126795d81a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:55:43.212054 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 456/912] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:55:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:55:43.502225 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c29815b-c335-437b-be16-48c82b5da318 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:43.629566 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c29815b-c335-437b-be16-48c82b5da318 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:43.666888 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe70129-eba8-423c-831b-fe5a15aae9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:43.668143 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe70129-eba8-423c-831b-fe5a15aae9f0 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:43.668143 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe70129-eba8-423c-831b-fe5a15aae9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:43.668304 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe70129-eba8-423c-831b-fe5a15aae9f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:43.681407 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe70129-eba8-423c-831b-fe5a15aae9f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:55:43.688048 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-5fe70129-eba8-423c-831b-fe5a15aae9f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 19:55:43.688048 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:55:43.688048 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:55:43.688048 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:55:43.688048 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:55:43.688048 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:55:43.688048 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:55:43.688048 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:55:43.688048 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:55:43.688048 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:55:43.688048 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:55:43.688048 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:55:43.688048 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:55:43.688048 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:55:43.688048 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:55:43.688048 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:55:43.689076 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:55:43.689076 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:55:43.689076 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:55:43.689076 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:55:43.689076 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:55:43.689076 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:55:43.689076 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:55:43.689076 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:55:43.689076 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:55:43.689076 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:55:43.689076 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:55:43.689076 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:55:43.689076 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:55:43.689076 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:55:43.689076 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:55:43.689076 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:55:43.689934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:55:43.689934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:55:43.689934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:55:43.689934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:55:43.689934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:55:43.689934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:55:43.689934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:55:43.689934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:55:43.689934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:55:43.689934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:55:43.689934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 19:55:43.689934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:55:43.691834 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 457/913] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Oct 23 19:55:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:55:43.698595 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22130a1-66c0-4a0c-b42e-1126795d81a0 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:43.748452 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:43.824475 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22130a1-66c0-4a0c-b42e-1126795d81a0 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:43.865269 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832dc09-1013-4a0a-9a4e-fb4ce84bcadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:43.865897 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832dc09-1013-4a0a-9a4e-fb4ce84bcadd None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:43.866170 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832dc09-1013-4a0a-9a4e-fb4ce84bcadd None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:43.866476 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0832dc09-1013-4a0a-9a4e-fb4ce84bcadd None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:43.867178 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0832dc09-1013-4a0a-9a4e-fb4ce84bcadd None admin] RBAC: Adding query filter params (name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:55:43.867478 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0832dc09-1013-4a0a-9a4e-fb4ce84bcadd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:43.869077 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0832dc09-1013-4a0a-9a4e-fb4ce84bcadd None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:43.877762 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 457/914] 10.222.0.13 () {56 vars in 1103 bytes} [Sun Oct 23 19:55:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:55:43.884161 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe70129-eba8-423c-831b-fe5a15aae9f0 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:43.938738 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:44.000202 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe70129-eba8-423c-831b-fe5a15aae9f0 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:44.049040 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f3484-77e7-42fa-9aeb-57f170759f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:44.049040 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f3484-77e7-42fa-9aeb-57f170759f65 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:44.049040 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f3484-77e7-42fa-9aeb-57f170759f65 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:44.049437 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f3484-77e7-42fa-9aeb-57f170759f65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:44.062722 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509f3484-77e7-42fa-9aeb-57f170759f65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:44.064499 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509f3484-77e7-42fa-9aeb-57f170759f65 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:44.088342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-509f3484-77e7-42fa-9aeb-57f170759f65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:55:44.088342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:55:44.088342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:55:44.088342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:55:44.088342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:55:44.088342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:55:44.088342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:55:44.088342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:55:44.088342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:55:44.088342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:55:44.088342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:55:44.088342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:55:44.088342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:55:44.088342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:55:44.088342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:55:44.088342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:55:44.089515 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:55:44.089515 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:55:44.089515 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:55:44.089515 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:55:44.089515 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:55:44.089515 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:55:44.089515 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:55:44.089515 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:55:44.089515 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:55:44.089515 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:55:44.089515 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:55:44.089515 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:55:44.089515 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:55:44.089515 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:55:44.089515 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:55:44.089515 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:55:44.089515 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:55:44.090608 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:55:44.090608 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:55:44.090608 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:55:44.090608 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 458/915] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:55:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:55:44.093383 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832dc09-1013-4a0a-9a4e-fb4ce84bcadd None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:44.205658 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:44.209331 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0832dc09-1013-4a0a-9a4e-fb4ce84bcadd None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:44.249012 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4f9e4e-6603-4326-a9a4-7d0d0b543927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:44.249637 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f9e4e-6603-4326-a9a4-7d0d0b543927 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:44.249927 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f9e4e-6603-4326-a9a4-7d0d0b543927 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:44.250227 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4f9e4e-6603-4326-a9a4-7d0d0b543927 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:44.250854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4f9e4e-6603-4326-a9a4-7d0d0b543927 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:55:44.251177 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4f9e4e-6603-4326-a9a4-7d0d0b543927 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:44.252609 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4f9e4e-6603-4326-a9a4-7d0d0b543927 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:44.264831 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 458/916] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:55:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:55:44.271019 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f3484-77e7-42fa-9aeb-57f170759f65 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:44.392852 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f3484-77e7-42fa-9aeb-57f170759f65 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:44.394417 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:44.435657 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6fae94-d093-4e08-8353-fb0cc147e5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:44.436394 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fae94-d093-4e08-8353-fb0cc147e5bc None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:44.436678 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fae94-d093-4e08-8353-fb0cc147e5bc None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:44.436951 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6fae94-d093-4e08-8353-fb0cc147e5bc None admin] PATH_INFO: `/v3/users/placement` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:44.445795 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6fae94-d093-4e08-8353-fb0cc147e5bc None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:44.447564 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6fae94-d093-4e08-8353-fb0cc147e5bc None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:44.461142 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-3f6fae94-d093-4e08-8353-fb0cc147e5bc None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 23 19:55:44.461142 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:55:44.461142 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:55:44.461142 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:55:44.461142 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:55:44.461142 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:55:44.461142 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:55:44.461142 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:55:44.461142 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:55:44.461142 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:55:44.461142 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:55:44.461142 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:55:44.461142 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 19:55:44.461142 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 19:55:44.461142 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 19:55:44.461142 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 19:55:44.462089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:55:44.462089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:55:44.462089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 19:55:44.462089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:55:44.462089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 19:55:44.462089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:55:44.462089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:55:44.462089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:55:44.462089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:55:44.462089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:55:44.462089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:55:44.462089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:55:44.462089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:55:44.462089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:55:44.462089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:55:44.462089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:55:44.462089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:55:44.462089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:55:44.463092 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:55:44.463092 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:55:44.463092 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 19:55:44.463092 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 19:55:44.463092 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 19:55:44.463092 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 19:55:44.463092 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 19:55:44.463092 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 19:55:44.463092 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 23 19:55:44.463092 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:55:44.463092 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 459/917] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:55:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:55:44.469042 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4f9e4e-6603-4326-a9a4-7d0d0b543927 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:44.594377 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4f9e4e-6603-4326-a9a4-7d0d0b543927 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:44.635115 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938aae52-e91c-4f4f-a483-3fb8cbcbad64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:44.635738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938aae52-e91c-4f4f-a483-3fb8cbcbad64 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:44.636023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938aae52-e91c-4f4f-a483-3fb8cbcbad64 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:44.636301 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938aae52-e91c-4f4f-a483-3fb8cbcbad64 None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:44.637042 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938aae52-e91c-4f4f-a483-3fb8cbcbad64 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:55:44.637344 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938aae52-e91c-4f4f-a483-3fb8cbcbad64 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:44.639330 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938aae52-e91c-4f4f-a483-3fb8cbcbad64 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:44.657793 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 459/918] 10.222.0.13 () {56 vars in 1143 bytes} [Sun Oct 23 19:55:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:55:44.663789 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6fae94-d093-4e08-8353-fb0cc147e5bc None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:44.797062 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6fae94-d093-4e08-8353-fb0cc147e5bc None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:44.832826 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f998c-fd74-4a45-b225-648b09d90687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:44.833690 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f998c-fd74-4a45-b225-648b09d90687 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:44.834126 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f998c-fd74-4a45-b225-648b09d90687 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:44.834580 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f998c-fd74-4a45-b225-648b09d90687 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:44.850401 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f998c-fd74-4a45-b225-648b09d90687 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:44.850401 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f998c-fd74-4a45-b225-648b09d90687 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:44.872152 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-560f998c-fd74-4a45-b225-648b09d90687 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:55:44.872152 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:55:44.872152 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:55:44.872152 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:55:44.872152 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:55:44.872152 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:55:44.872152 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:55:44.872152 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:55:44.872152 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:55:44.872152 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:55:44.872152 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:55:44.872152 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:55:44.872152 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:55:44.872152 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:55:44.872152 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:55:44.872152 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:55:44.873933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:55:44.873933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:55:44.873933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:55:44.873933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:55:44.873933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:55:44.873933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:55:44.873933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:55:44.873933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:55:44.873933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:55:44.873933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:55:44.873933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:55:44.873933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:55:44.873933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:55:44.873933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:55:44.873933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:55:44.873933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:55:44.873933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:55:44.875105 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:55:44.875105 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:55:44.875105 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:55:44.876924 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 460/919] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:55:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:55:44.883719 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938aae52-e91c-4f4f-a483-3fb8cbcbad64 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:44.904367 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:44.998873 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938aae52-e91c-4f4f-a483-3fb8cbcbad64 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:45.040462 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8452b3c-3604-401f-a743-b785d4ec85cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:45.041082 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8452b3c-3604-401f-a743-b785d4ec85cb None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:45.041371 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8452b3c-3604-401f-a743-b785d4ec85cb None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:45.041714 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8452b3c-3604-401f-a743-b785d4ec85cb None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:45.042424 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8452b3c-3604-401f-a743-b785d4ec85cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:55:45.042752 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8452b3c-3604-401f-a743-b785d4ec85cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:45.044168 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8452b3c-3604-401f-a743-b785d4ec85cb None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:45.056686 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 460/920] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:55:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:55:45.062493 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f998c-fd74-4a45-b225-648b09d90687 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:45.080709 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:45.177584 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f998c-fd74-4a45-b225-648b09d90687 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:45.228655 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef7872-9131-4d7b-b748-030dcdbb9ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:45.229309 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef7872-9131-4d7b-b748-030dcdbb9ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:45.229716 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef7872-9131-4d7b-b748-030dcdbb9ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:45.229891 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef7872-9131-4d7b-b748-030dcdbb9ec8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:45.238216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ef7872-9131-4d7b-b748-030dcdbb9ec8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:45.239685 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ef7872-9131-4d7b-b748-030dcdbb9ec8 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:45.251491 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-27ef7872-9131-4d7b-b748-030dcdbb9ec8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:55:45.251491 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:55:45.251491 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:55:45.251491 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:55:45.251491 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:55:45.251491 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:55:45.251491 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:55:45.251491 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:55:45.251491 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:55:45.251491 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:55:45.251491 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:55:45.251491 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:55:45.251491 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 19:55:45.251491 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 19:55:45.251491 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 19:55:45.252428 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 19:55:45.252428 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:55:45.252428 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:55:45.252428 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:55:45.252428 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:55:45.252428 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:55:45.252428 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:55:45.252428 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:55:45.252428 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:55:45.252428 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:55:45.252428 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:55:45.252428 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:55:45.252428 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:55:45.252428 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:55:45.252428 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:55:45.252428 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:55:45.252428 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:55:45.253390 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 19:55:45.253390 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 19:55:45.253390 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 19:55:45.253390 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 19:55:45.253390 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 19:55:45.253390 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 19:55:45.253390 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:55:45.253390 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:55:45.253390 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 461/921] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:55:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:55:45.259649 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8452b3c-3604-401f-a743-b785d4ec85cb None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:45.348584 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:45.373696 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8452b3c-3604-401f-a743-b785d4ec85cb None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:45.415688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab2417-e4c7-4b62-9c6c-8337fa8c779d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:45.416310 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab2417-e4c7-4b62-9c6c-8337fa8c779d None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:45.416649 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab2417-e4c7-4b62-9c6c-8337fa8c779d None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:45.416936 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab2417-e4c7-4b62-9c6c-8337fa8c779d None admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:45.417738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eab2417-e4c7-4b62-9c6c-8337fa8c779d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:55:45.418050 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eab2417-e4c7-4b62-9c6c-8337fa8c779d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:45.419659 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eab2417-e4c7-4b62-9c6c-8337fa8c779d None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:45.434280 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 461/922] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Oct 23 19:55:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:55:45.442085 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef7872-9131-4d7b-b748-030dcdbb9ec8 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:45.531703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:45.564034 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef7872-9131-4d7b-b748-030dcdbb9ec8 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:45.609018 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e25384-e87b-4803-92a1-9551c1d1dd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:45.609782 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e25384-e87b-4803-92a1-9551c1d1dd8e None admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:45.610083 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e25384-e87b-4803-92a1-9551c1d1dd8e None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:45.610482 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e25384-e87b-4803-92a1-9551c1d1dd8e None admin] PATH_INFO: `/v3/projects/d40947427f5f43e7b6c82e3605737767/users/312489c49a304ca9aacebd9843a4b65f/roles/5e0524ede3ee479c923c7b609a0ca402` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:45.629164 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e25384-e87b-4803-92a1-9551c1d1dd8e None admin] RBAC: Authorizing `identity:create_grant(project_id=d40947427f5f43e7b6c82e3605737767, user_id=312489c49a304ca9aacebd9843a4b65f, role_id=5e0524ede3ee479c923c7b609a0ca402)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:45.630928 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e25384-e87b-4803-92a1-9551c1d1dd8e None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:45.648172 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 462/923] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Oct 23 19:55:45 2022] PUT /identity/v3/projects/d40947427f5f43e7b6c82e3605737767/users/312489c49a304ca9aacebd9843a4b65f/roles/5e0524ede3ee479c923c7b609a0ca402 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 19:55:48.007566 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ffe4b-43de-4b59-8e3c-4914953708c0 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:48.008128 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ffe4b-43de-4b59-8e3c-4914953708c0 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:48.008128 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788ffe4b-43de-4b59-8e3c-4914953708c0 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:48.009680 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 462/924] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:55:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:55:48.018287 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d5c3d-645f-4db6-b377-f10a1b3ae17f None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:48.018287 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d5c3d-645f-4db6-b377-f10a1b3ae17f None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:48.018287 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80d5c3d-645f-4db6-b377-f10a1b3ae17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:48.035966 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-e80d5c3d-645f-4db6-b377-f10a1b3ae17f None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:55:48.037306 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-e80d5c3d-645f-4db6-b377-f10a1b3ae17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:55:48.105687 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 463/925] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:55:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:55:48.389111 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ffe4b-43de-4b59-8e3c-4914953708c0 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:48.503344 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788ffe4b-43de-4b59-8e3c-4914953708c0 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:48.543353 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb8edde-4e8a-41da-be38-b72c0c2b5f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:48.544077 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8edde-4e8a-41da-be38-b72c0c2b5f32 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:48.544470 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8edde-4e8a-41da-be38-b72c0c2b5f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:48.544876 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8edde-4e8a-41da-be38-b72c0c2b5f32 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:48.551785 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:48.556298 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb8edde-4e8a-41da-be38-b72c0c2b5f32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:55:48.563807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-2bb8edde-4e8a-41da-be38-b72c0c2b5f32 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 19:55:48.563807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:55:48.563807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:55:48.563807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:55:48.563807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:55:48.563807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:55:48.563807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:55:48.563807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:55:48.563807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:55:48.563807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:55:48.563807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:55:48.563807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:55:48.563807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:55:48.563807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:55:48.563807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:55:48.563807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:55:48.565073 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:55:48.565073 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:55:48.565073 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:55:48.565073 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:55:48.565073 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:55:48.565073 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:55:48.565073 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:55:48.565073 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:55:48.565073 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:55:48.565073 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:55:48.565073 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:55:48.565073 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:55:48.565073 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:55:48.565073 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:55:48.565073 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:55:48.565073 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:55:48.566391 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:55:48.566391 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:55:48.566391 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:55:48.566391 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:55:48.566391 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:55:48.566391 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:55:48.566391 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:55:48.566391 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:55:48.566391 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:55:48.566391 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:55:48.566391 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 19:55:48.566391 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:55:48.566391 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 463/926] 10.222.0.13 () {56 vars in 1102 bytes} [Sun Oct 23 19:55:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:55:48.572425 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d5c3d-645f-4db6-b377-f10a1b3ae17f None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:48.634702 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:48.683819 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80d5c3d-645f-4db6-b377-f10a1b3ae17f None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:48.721862 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb2ed9-ed09-4dff-9678-e917fbf36d79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:48.722549 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb2ed9-ed09-4dff-9678-e917fbf36d79 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:48.722828 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb2ed9-ed09-4dff-9678-e917fbf36d79 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:48.723099 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deb2ed9-ed09-4dff-9678-e917fbf36d79 None admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:48.723813 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb2ed9-ed09-4dff-9678-e917fbf36d79 None admin] RBAC: Adding query filter params (name=service) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:55:48.724137 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb2ed9-ed09-4dff-9678-e917fbf36d79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:48.725489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deb2ed9-ed09-4dff-9678-e917fbf36d79 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:48.734123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 464/927] 10.222.0.13 () {56 vars in 1103 bytes} [Sun Oct 23 19:55:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:55:48.740410 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb8edde-4e8a-41da-be38-b72c0c2b5f32 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:48.857347 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb8edde-4e8a-41da-be38-b72c0c2b5f32 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:48.898105 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf249355-8584-4b50-87b0-b7449f58a77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:48.898788 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf249355-8584-4b50-87b0-b7449f58a77e None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:48.899066 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf249355-8584-4b50-87b0-b7449f58a77e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:48.899347 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf249355-8584-4b50-87b0-b7449f58a77e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:48.910894 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf249355-8584-4b50-87b0-b7449f58a77e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:48.912337 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf249355-8584-4b50-87b0-b7449f58a77e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:48.927759 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-bf249355-8584-4b50-87b0-b7449f58a77e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:55:48.927759 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:55:48.927759 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:55:48.927759 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:55:48.927759 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:55:48.927759 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:55:48.927759 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:55:48.927759 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:55:48.927759 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:55:48.927759 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:55:48.927759 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:55:48.927759 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:55:48.927759 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:55:48.927759 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:55:48.927759 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:55:48.927759 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:55:48.929054 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:55:48.929054 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:55:48.929054 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:55:48.929054 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:55:48.929054 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:55:48.929054 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:55:48.929054 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:55:48.929054 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:55:48.929054 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:55:48.929054 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:55:48.929054 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:55:48.929054 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:55:48.929054 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:55:48.929054 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:55:48.929054 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:55:48.929054 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:55:48.929054 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:55:48.929954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:55:48.929954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:55:48.929954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:55:48.929954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 464/928] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:55:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:55:48.934807 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb2ed9-ed09-4dff-9678-e917fbf36d79 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:49.046901 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deb2ed9-ed09-4dff-9678-e917fbf36d79 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:49.063885 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:49.083990 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab637b9e-0505-4ace-b059-2a42911eebec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:49.084594 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab637b9e-0505-4ace-b059-2a42911eebec None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:49.084872 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab637b9e-0505-4ace-b059-2a42911eebec None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:49.085139 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab637b9e-0505-4ace-b059-2a42911eebec None admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:49.085810 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab637b9e-0505-4ace-b059-2a42911eebec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:55:49.086148 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab637b9e-0505-4ace-b059-2a42911eebec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:49.087535 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab637b9e-0505-4ace-b059-2a42911eebec None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:49.100421 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 465/929] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:55:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:55:49.107649 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf249355-8584-4b50-87b0-b7449f58a77e None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:49.160733 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:49.214164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf249355-8584-4b50-87b0-b7449f58a77e None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:49.251351 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20929fb-122f-4b5d-bab1-09cb20cf95a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:49.251986 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20929fb-122f-4b5d-bab1-09cb20cf95a6 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:49.252263 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20929fb-122f-4b5d-bab1-09cb20cf95a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:49.252560 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20929fb-122f-4b5d-bab1-09cb20cf95a6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:49.261697 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20929fb-122f-4b5d-bab1-09cb20cf95a6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:49.263788 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20929fb-122f-4b5d-bab1-09cb20cf95a6 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:49.277737 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-b20929fb-122f-4b5d-bab1-09cb20cf95a6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 23 19:55:49.277737 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:55:49.277737 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:55:49.277737 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:55:49.277737 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:55:49.277737 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:55:49.277737 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:55:49.277737 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:55:49.277737 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:55:49.277737 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:55:49.277737 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:55:49.277737 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:55:49.277737 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 19:55:49.277737 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 19:55:49.277737 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 19:55:49.277737 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 19:55:49.279545 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:55:49.279545 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:55:49.279545 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 19:55:49.279545 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:55:49.279545 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 19:55:49.279545 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:55:49.279545 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:55:49.279545 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:55:49.279545 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:55:49.279545 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:55:49.279545 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:55:49.279545 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:55:49.279545 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:55:49.279545 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:55:49.279545 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:55:49.279545 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:55:49.279545 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:55:49.279545 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:55:49.280702 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:55:49.280702 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:55:49.280702 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 19:55:49.280702 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 19:55:49.280702 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 19:55:49.280702 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 19:55:49.280702 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 19:55:49.280702 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 19:55:49.280702 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 23 19:55:49.280702 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:55:49.284401 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 465/930] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:55:49 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:55:49.285658 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab637b9e-0505-4ace-b059-2a42911eebec None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:49.394499 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab637b9e-0505-4ace-b059-2a42911eebec None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:49.429295 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeef34d-0059-4ace-b19d-0542b11756fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:49.429900 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeef34d-0059-4ace-b19d-0542b11756fe None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:49.430179 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeef34d-0059-4ace-b19d-0542b11756fe None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:49.430489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caeef34d-0059-4ace-b19d-0542b11756fe None admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:49.431244 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeef34d-0059-4ace-b19d-0542b11756fe None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:55:49.431528 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeef34d-0059-4ace-b19d-0542b11756fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:49.432944 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caeef34d-0059-4ace-b19d-0542b11756fe None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:49.450701 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 466/931] 10.222.0.13 () {56 vars in 1143 bytes} [Sun Oct 23 19:55:49 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 19:55:49.456476 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20929fb-122f-4b5d-bab1-09cb20cf95a6 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:49.567506 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20929fb-122f-4b5d-bab1-09cb20cf95a6 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:49.578843 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:49.604161 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6fe9d-f787-409e-81e1-9b2b4a1f9612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:49.604773 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6fe9d-f787-409e-81e1-9b2b4a1f9612 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:49.605057 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6fe9d-f787-409e-81e1-9b2b4a1f9612 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:49.605325 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac6fe9d-f787-409e-81e1-9b2b4a1f9612 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:49.680932 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac6fe9d-f787-409e-81e1-9b2b4a1f9612 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:49.682409 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac6fe9d-f787-409e-81e1-9b2b4a1f9612 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:49.797880 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 466/932] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Oct 23 19:55:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 23 19:55:49.803745 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeef34d-0059-4ace-b19d-0542b11756fe None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:49.840559 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:49.915209 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caeef34d-0059-4ace-b19d-0542b11756fe None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:49.956404 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16663311-68ea-4cfa-83ad-0b9867b466dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:49.957031 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16663311-68ea-4cfa-83ad-0b9867b466dd None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:49.957313 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16663311-68ea-4cfa-83ad-0b9867b466dd None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:49.957710 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16663311-68ea-4cfa-83ad-0b9867b466dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:49.969122 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16663311-68ea-4cfa-83ad-0b9867b466dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:49.970583 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16663311-68ea-4cfa-83ad-0b9867b466dd None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:49.986910 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-16663311-68ea-4cfa-83ad-0b9867b466dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:55:49.986910 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:55:49.986910 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:55:49.986910 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:55:49.986910 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:55:49.986910 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:55:49.986910 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:55:49.986910 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:55:49.986910 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:55:49.986910 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:55:49.986910 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:55:49.986910 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:55:49.986910 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:55:49.986910 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:55:49.986910 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:55:49.986910 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:55:49.988014 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:55:49.988014 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:55:49.988014 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:55:49.988014 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:55:49.988014 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:55:49.988014 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:55:49.988014 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:55:49.988014 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:55:49.988014 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:55:49.988014 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:55:49.988014 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:55:49.988014 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:55:49.988014 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:55:49.988014 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:55:49.988014 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:55:49.988014 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:55:49.988014 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:55:49.988930 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:55:49.988930 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:55:49.988930 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:55:49.988930 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 467/933] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:55:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:55:49.995062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6fe9d-f787-409e-81e1-9b2b4a1f9612 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:50.013261 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:50.109388 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac6fe9d-f787-409e-81e1-9b2b4a1f9612 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:50.147031 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c4e86-f5d0-4ab1-9201-537f6c88fecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:50.147674 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c4e86-f5d0-4ab1-9201-537f6c88fecd None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:50.147991 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c4e86-f5d0-4ab1-9201-537f6c88fecd None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:50.148437 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17c4e86-f5d0-4ab1-9201-537f6c88fecd None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:50.149250 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c4e86-f5d0-4ab1-9201-537f6c88fecd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:55:50.149553 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c4e86-f5d0-4ab1-9201-537f6c88fecd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:50.151251 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17c4e86-f5d0-4ab1-9201-537f6c88fecd None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:50.165600 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 467/934] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:55:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:55:50.171501 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16663311-68ea-4cfa-83ad-0b9867b466dd None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:50.263749 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:50.280791 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16663311-68ea-4cfa-83ad-0b9867b466dd None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:50.320773 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887c6f8-bdf8-47c5-b700-144f1aad2bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:50.321492 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887c6f8-bdf8-47c5-b700-144f1aad2bfb None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:50.321767 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887c6f8-bdf8-47c5-b700-144f1aad2bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:50.322053 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887c6f8-bdf8-47c5-b700-144f1aad2bfb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:50.330687 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887c6f8-bdf8-47c5-b700-144f1aad2bfb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:50.332928 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a887c6f8-bdf8-47c5-b700-144f1aad2bfb None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:50.348096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-a887c6f8-bdf8-47c5-b700-144f1aad2bfb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:55:50.348096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:55:50.348096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:55:50.348096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:55:50.348096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:55:50.348096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:55:50.348096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:55:50.348096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:55:50.348096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:55:50.348096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:55:50.348096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:55:50.348096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:55:50.348096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 19:55:50.348096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 19:55:50.348096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 19:55:50.349181 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 19:55:50.349181 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:55:50.349181 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:55:50.349181 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:55:50.349181 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:55:50.349181 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:55:50.349181 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:55:50.349181 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:55:50.349181 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:55:50.349181 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:55:50.349181 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:55:50.349181 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:55:50.349181 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:55:50.349181 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:55:50.349181 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:55:50.349181 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:55:50.349181 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:55:50.350062 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 19:55:50.350062 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 19:55:50.350062 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 19:55:50.350062 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 19:55:50.350062 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 19:55:50.350062 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 19:55:50.350062 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:55:50.350062 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:55:50.350062 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 468/935] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:55:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:55:50.355220 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c4e86-f5d0-4ab1-9201-537f6c88fecd None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:50.439480 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:50.479031 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17c4e86-f5d0-4ab1-9201-537f6c88fecd None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:50.520906 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f087e54-3f9c-4396-84c3-e6df59863b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:50.521531 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f087e54-3f9c-4396-84c3-e6df59863b4b None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:50.521814 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f087e54-3f9c-4396-84c3-e6df59863b4b None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:50.522090 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f087e54-3f9c-4396-84c3-e6df59863b4b None admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:50.522827 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f087e54-3f9c-4396-84c3-e6df59863b4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:55:50.523137 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f087e54-3f9c-4396-84c3-e6df59863b4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:50.524687 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f087e54-3f9c-4396-84c3-e6df59863b4b None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:50.537863 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 468/936] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Oct 23 19:55:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:55:50.544108 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887c6f8-bdf8-47c5-b700-144f1aad2bfb None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:50.651630 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887c6f8-bdf8-47c5-b700-144f1aad2bfb None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:50.691415 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055d005e-4041-4e88-b8e2-702a0fedcb57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:50.691947 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d005e-4041-4e88-b8e2-702a0fedcb57 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:50.692228 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d005e-4041-4e88-b8e2-702a0fedcb57 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:50.692499 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055d005e-4041-4e88-b8e2-702a0fedcb57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:50.693220 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055d005e-4041-4e88-b8e2-702a0fedcb57 None admin] RBAC: Adding query filter params (role.id=5e0524ede3ee479c923c7b609a0ca402, scope.project.id=d40947427f5f43e7b6c82e3605737767, user.id=312489c49a304ca9aacebd9843a4b65f) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:55:50.693528 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055d005e-4041-4e88-b8e2-702a0fedcb57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:50.694976 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055d005e-4041-4e88-b8e2-702a0fedcb57 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:50.701253 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 469/937] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Oct 23 19:55:50 2022] GET /identity/v3/role_assignments?user.id=312489c49a304ca9aacebd9843a4b65f&scope.project.id=d40947427f5f43e7b6c82e3605737767&role.id=5e0524ede3ee479c923c7b609a0ca402 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:55:53.017576 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb680d-951d-4dee-a260-c76cac15385a None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:53.018134 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb680d-951d-4dee-a260-c76cac15385a None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:53.018134 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eb680d-951d-4dee-a260-c76cac15385a None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:53.019644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 469/938] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:55:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:55:53.024990 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d8f08b-4157-473e-99cb-63cd0fbbb41b None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:53.025425 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d8f08b-4157-473e-99cb-63cd0fbbb41b None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:53.025617 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d8f08b-4157-473e-99cb-63cd0fbbb41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:53.046044 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-16d8f08b-4157-473e-99cb-63cd0fbbb41b None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:55:53.047269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-16d8f08b-4157-473e-99cb-63cd0fbbb41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:55:53.050593 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:53.125918 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 470/939] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:55:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:55:53.420331 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb680d-951d-4dee-a260-c76cac15385a None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:53.525307 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eb680d-951d-4dee-a260-c76cac15385a None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:53.555805 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:53.562633 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a40b3-16eb-4781-9725-343aa55b1963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:53.563497 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a40b3-16eb-4781-9725-343aa55b1963 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:53.564073 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a40b3-16eb-4781-9725-343aa55b1963 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:53.564605 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a40b3-16eb-4781-9725-343aa55b1963 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:53.575717 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46a40b3-16eb-4781-9725-343aa55b1963 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:55:53.580589 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-c46a40b3-16eb-4781-9725-343aa55b1963 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:55:53.580589 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:55:53.580589 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:55:53.580589 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:55:53.580589 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:55:53.580589 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:55:53.580589 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:55:53.580589 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:55:53.580589 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:55:53.580589 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:55:53.580589 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:55:53.580589 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:55:53.580589 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:55:53.580589 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:55:53.580589 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:55:53.580589 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:55:53.582045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:55:53.582045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:55:53.582045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:55:53.582045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:55:53.582045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:55:53.582045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:55:53.582045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:55:53.582045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:55:53.582045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:55:53.582045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:55:53.582045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:55:53.582045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:55:53.582045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:55:53.582045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:55:53.582045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:55:53.582045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:55:53.583254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:55:53.583254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:55:53.583254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:55:53.583254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:55:53.583254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:55:53.583254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:55:53.583254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:55:53.583254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:55:53.583254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:55:53.583254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:55:53.583254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:55:53.583254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:55:53.583254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 470/940] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Oct 23 19:55:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:55:53.587751 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d8f08b-4157-473e-99cb-63cd0fbbb41b None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:53.692023 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d8f08b-4157-473e-99cb-63cd0fbbb41b None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:53.725622 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4187ee1-90e0-4908-bc2b-ef57ab8791f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:53.726234 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4187ee1-90e0-4908-bc2b-ef57ab8791f8 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:53.726537 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4187ee1-90e0-4908-bc2b-ef57ab8791f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:53.726806 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4187ee1-90e0-4908-bc2b-ef57ab8791f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:53.727471 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4187ee1-90e0-4908-bc2b-ef57ab8791f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:55:53.727772 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4187ee1-90e0-4908-bc2b-ef57ab8791f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:53.729128 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4187ee1-90e0-4908-bc2b-ef57ab8791f8 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:53.737712 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 471/941] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:55:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:55:53.743565 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a40b3-16eb-4781-9725-343aa55b1963 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:53.861902 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a40b3-16eb-4781-9725-343aa55b1963 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:53.900442 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549fc543-f573-491c-a3f4-eaa1a60dcc5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:53.901073 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fc543-f573-491c-a3f4-eaa1a60dcc5d None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:53.901350 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fc543-f573-491c-a3f4-eaa1a60dcc5d None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:53.901620 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549fc543-f573-491c-a3f4-eaa1a60dcc5d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:53.914141 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549fc543-f573-491c-a3f4-eaa1a60dcc5d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:53.915995 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549fc543-f573-491c-a3f4-eaa1a60dcc5d None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:53.932848 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-549fc543-f573-491c-a3f4-eaa1a60dcc5d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:55:53.932848 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:55:53.932848 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:55:53.932848 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:55:53.932848 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:55:53.932848 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:55:53.932848 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:55:53.932848 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:55:53.932848 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:55:53.932848 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:55:53.932848 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:55:53.932848 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:55:53.932848 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:55:53.932848 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:55:53.932848 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:55:53.932848 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:55:53.933854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:55:53.933854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:55:53.933854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:55:53.933854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:55:53.933854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:55:53.933854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:55:53.933854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:55:53.933854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:55:53.933854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:55:53.933854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:55:53.933854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:55:53.933854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:55:53.933854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:55:53.933854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:55:53.933854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:55:53.933854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:55:53.933854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:55:53.934788 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:55:53.934788 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:55:53.934788 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:55:53.934788 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 471/942] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:55:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:55:53.940896 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4187ee1-90e0-4908-bc2b-ef57ab8791f8 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:53.944831 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:54.064270 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4187ee1-90e0-4908-bc2b-ef57ab8791f8 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:54.102921 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5a244-1179-42e2-af08-1ba32cc5884f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:54.104056 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5a244-1179-42e2-af08-1ba32cc5884f None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:54.104412 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5a244-1179-42e2-af08-1ba32cc5884f None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:54.104753 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe5a244-1179-42e2-af08-1ba32cc5884f None admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:54.105579 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe5a244-1179-42e2-af08-1ba32cc5884f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:55:54.106103 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe5a244-1179-42e2-af08-1ba32cc5884f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:54.108083 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe5a244-1179-42e2-af08-1ba32cc5884f None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:54.121302 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 472/943] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:55:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:55:54.127089 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549fc543-f573-491c-a3f4-eaa1a60dcc5d None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:54.181313 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:54.248474 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549fc543-f573-491c-a3f4-eaa1a60dcc5d None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:54.285994 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e91443-89b5-490a-b3bf-a0636f9d751e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:54.286914 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e91443-89b5-490a-b3bf-a0636f9d751e None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:54.287386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e91443-89b5-490a-b3bf-a0636f9d751e None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:54.287643 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e91443-89b5-490a-b3bf-a0636f9d751e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:54.296698 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e91443-89b5-490a-b3bf-a0636f9d751e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:54.298166 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e91443-89b5-490a-b3bf-a0636f9d751e None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:54.309301 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-e4e91443-89b5-490a-b3bf-a0636f9d751e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 23 19:55:54.309301 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:55:54.309301 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:55:54.309301 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:55:54.309301 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:55:54.309301 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:55:54.309301 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:55:54.309301 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:55:54.309301 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:55:54.309301 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:55:54.309301 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:55:54.309301 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:55:54.309301 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 19:55:54.309301 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 19:55:54.309301 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 19:55:54.309301 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 19:55:54.315794 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:55:54.315794 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:55:54.315794 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 19:55:54.315794 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:55:54.315794 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 19:55:54.315794 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:55:54.315794 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:55:54.315794 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:55:54.315794 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:55:54.315794 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:55:54.315794 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:55:54.315794 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:55:54.315794 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:55:54.315794 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:55:54.315794 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:55:54.315794 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:55:54.315794 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:55:54.315794 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:55:54.317253 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:55:54.317253 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:55:54.317253 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 19:55:54.317253 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 19:55:54.317253 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 19:55:54.317253 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 19:55:54.317253 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 19:55:54.317253 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 19:55:54.317253 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 23 19:55:54.317253 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:55:54.317253 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 472/944] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:55:54 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:55:54.317980 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5a244-1179-42e2-af08-1ba32cc5884f None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:54.384621 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:54.430322 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe5a244-1179-42e2-af08-1ba32cc5884f None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:54.464344 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd5b67-c441-461c-9740-3927e282f3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:54.464953 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd5b67-c441-461c-9740-3927e282f3ee None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:54.465242 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd5b67-c441-461c-9740-3927e282f3ee None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:54.465503 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fd5b67-c441-461c-9740-3927e282f3ee None admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:54.466821 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd5b67-c441-461c-9740-3927e282f3ee None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:55:54.467133 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd5b67-c441-461c-9740-3927e282f3ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:54.468666 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fd5b67-c441-461c-9740-3927e282f3ee None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:54.486342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 473/945] 10.222.0.13 () {56 vars in 1143 bytes} [Sun Oct 23 19:55:54 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:55:54.492501 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e91443-89b5-490a-b3bf-a0636f9d751e None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:54.602347 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e91443-89b5-490a-b3bf-a0636f9d751e None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:54.605115 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:54.642556 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa1644b-691a-4120-887b-c29dfbe9c87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:54.643290 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1644b-691a-4120-887b-c29dfbe9c87c None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:54.644099 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1644b-691a-4120-887b-c29dfbe9c87c None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:54.644099 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa1644b-691a-4120-887b-c29dfbe9c87c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:54.656566 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa1644b-691a-4120-887b-c29dfbe9c87c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:54.658430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa1644b-691a-4120-887b-c29dfbe9c87c None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:54.674850 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-faa1644b-691a-4120-887b-c29dfbe9c87c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:55:54.674850 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:55:54.674850 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:55:54.674850 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:55:54.674850 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:55:54.674850 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:55:54.674850 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:55:54.674850 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:55:54.674850 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:55:54.674850 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:55:54.674850 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:55:54.674850 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:55:54.674850 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:55:54.674850 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:55:54.674850 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:55:54.674850 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:55:54.676411 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:55:54.676411 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:55:54.676411 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:55:54.676411 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:55:54.676411 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:55:54.676411 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:55:54.676411 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:55:54.676411 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:55:54.676411 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:55:54.676411 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:55:54.676411 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:55:54.676411 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:55:54.676411 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:55:54.676411 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:55:54.676411 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:55:54.676411 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:55:54.676411 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:55:54.677650 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:55:54.677650 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:55:54.677650 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:55:54.679207 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 473/946] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:55:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:55:54.686071 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd5b67-c441-461c-9740-3927e282f3ee None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:54.806743 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fd5b67-c441-461c-9740-3927e282f3ee None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:54.828388 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:54.843326 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e53bb52-7aa9-4440-a21e-28ee765d31d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:54.843944 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53bb52-7aa9-4440-a21e-28ee765d31d6 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:54.844225 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53bb52-7aa9-4440-a21e-28ee765d31d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:54.844498 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e53bb52-7aa9-4440-a21e-28ee765d31d6 None admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:54.845137 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e53bb52-7aa9-4440-a21e-28ee765d31d6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:55:54.845441 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e53bb52-7aa9-4440-a21e-28ee765d31d6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:54.846844 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e53bb52-7aa9-4440-a21e-28ee765d31d6 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:54.860436 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 474/947] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:55:54.866396 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa1644b-691a-4120-887b-c29dfbe9c87c None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:54.980641 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa1644b-691a-4120-887b-c29dfbe9c87c None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:55.024204 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072b59f-9e79-45b9-b4b3-d100a34f2825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:55.024204 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072b59f-9e79-45b9-b4b3-d100a34f2825 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:55.024443 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072b59f-9e79-45b9-b4b3-d100a34f2825 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:55.024633 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2072b59f-9e79-45b9-b4b3-d100a34f2825 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:55.032310 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072b59f-9e79-45b9-b4b3-d100a34f2825 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:55.033727 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2072b59f-9e79-45b9-b4b3-d100a34f2825 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:55.045426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-2072b59f-9e79-45b9-b4b3-d100a34f2825 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:55:55.045426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:55:55.045426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:55:55.045426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:55:55.045426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:55:55.045426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:55:55.045426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:55:55.045426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:55:55.045426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:55:55.045426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:55:55.045426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:55:55.045426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:55:55.045426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 19:55:55.045426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 19:55:55.045426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 19:55:55.046536 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 19:55:55.046536 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:55:55.046536 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:55:55.046536 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:55:55.046536 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:55:55.046536 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:55:55.046536 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:55:55.046536 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:55:55.046536 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:55:55.046536 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:55:55.046536 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:55:55.046536 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:55:55.046536 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:55:55.046536 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:55:55.046536 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:55:55.046536 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:55:55.046536 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:55:55.047799 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 19:55:55.047799 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 19:55:55.047799 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 19:55:55.047799 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 19:55:55.047799 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 19:55:55.047799 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 19:55:55.047799 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:55:55.047799 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:55:55.047799 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 474/948] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:55:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:55:55.052452 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e53bb52-7aa9-4440-a21e-28ee765d31d6 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:55.163342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e53bb52-7aa9-4440-a21e-28ee765d31d6 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:55.202587 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7cc2b0f-02a0-4311-8bbf-e9ececadc698 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:55.203197 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc2b0f-02a0-4311-8bbf-e9ececadc698 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:55.203474 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc2b0f-02a0-4311-8bbf-e9ececadc698 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:55.203741 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7cc2b0f-02a0-4311-8bbf-e9ececadc698 None admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:55.204484 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc2b0f-02a0-4311-8bbf-e9ececadc698 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:55:55.204783 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc2b0f-02a0-4311-8bbf-e9ececadc698 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:55.206351 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7cc2b0f-02a0-4311-8bbf-e9ececadc698 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:55.220034 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 475/949] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Oct 23 19:55:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:55:55.226841 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072b59f-9e79-45b9-b4b3-d100a34f2825 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:55.243254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:55.334996 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2072b59f-9e79-45b9-b4b3-d100a34f2825 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:55.370528 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ce6251-cd7b-49e2-8f2f-239c81ef448f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:55.371244 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce6251-cd7b-49e2-8f2f-239c81ef448f None admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:55.371558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce6251-cd7b-49e2-8f2f-239c81ef448f None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:55.371849 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce6251-cd7b-49e2-8f2f-239c81ef448f None admin] PATH_INFO: `/v3/projects/d40947427f5f43e7b6c82e3605737767/users/312489c49a304ca9aacebd9843a4b65f/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:55.373933 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce6251-cd7b-49e2-8f2f-239c81ef448f None admin] RBAC: Authorizing `identity:create_grant(project_id=d40947427f5f43e7b6c82e3605737767, user_id=312489c49a304ca9aacebd9843a4b65f, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:55.375533 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ce6251-cd7b-49e2-8f2f-239c81ef448f None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:55.388285 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 475/950] 10.222.0.13 () {58 vars in 1439 bytes} [Sun Oct 23 19:55:55 2022] PUT /identity/v3/projects/d40947427f5f43e7b6c82e3605737767/users/312489c49a304ca9aacebd9843a4b65f/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 19:55:57.638613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c0750-0dad-47a5-aa92-e7e3d1f014e6 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:57.639157 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c0750-0dad-47a5-aa92-e7e3d1f014e6 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:57.639157 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c0750-0dad-47a5-aa92-e7e3d1f014e6 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:57.640671 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 476/951] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:55:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:55:57.646560 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efd9f3-d0f0-428f-a91a-1515f52c79f1 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:57.646987 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efd9f3-d0f0-428f-a91a-1515f52c79f1 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:57.647075 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79efd9f3-d0f0-428f-a91a-1515f52c79f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:57.667047 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-79efd9f3-d0f0-428f-a91a-1515f52c79f1 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:55:57.668454 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-79efd9f3-d0f0-428f-a91a-1515f52c79f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:55:57.747100 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 476/952] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:55:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:55:58.048710 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c0750-0dad-47a5-aa92-e7e3d1f014e6 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:58.097043 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:58.163408 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c0750-0dad-47a5-aa92-e7e3d1f014e6 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:58.201950 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99ccef-1666-43c7-8ccd-d2f1cf4359e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:58.202715 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99ccef-1666-43c7-8ccd-d2f1cf4359e2 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:58.202994 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99ccef-1666-43c7-8ccd-d2f1cf4359e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:58.203262 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e99ccef-1666-43c7-8ccd-d2f1cf4359e2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:58.213191 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e99ccef-1666-43c7-8ccd-d2f1cf4359e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 19:55:58.218548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-9e99ccef-1666-43c7-8ccd-d2f1cf4359e2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:55:58.218548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:55:58.218548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:55:58.218548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:55:58.218548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:55:58.218548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:55:58.218548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:55:58.218548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:55:58.218548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:55:58.218548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:55:58.218548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:55:58.218548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:55:58.218548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 19:55:58.218548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 19:55:58.218548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 19:55:58.218548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 19:55:58.219483 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 19:55:58.219483 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 19:55:58.219483 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 19:55:58.219483 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 19:55:58.219483 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:55:58.219483 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:55:58.219483 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:55:58.219483 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:55:58.219483 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:55:58.219483 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:55:58.219483 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:55:58.219483 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:55:58.219483 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:55:58.219483 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:55:58.219483 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:55:58.219483 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:55:58.220341 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:55:58.220341 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:55:58.220341 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:55:58.220341 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:55:58.220341 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 19:55:58.220341 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 19:55:58.220341 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 19:55:58.220341 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 19:55:58.220341 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 19:55:58.220341 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 19:55:58.220341 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 19:55:58.220341 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:55:58.220341 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 477/953] 10.222.0.13 () {56 vars in 1096 bytes} [Sun Oct 23 19:55:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:55:58.225653 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79efd9f3-d0f0-428f-a91a-1515f52c79f1 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:58.228806 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:58.336681 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79efd9f3-d0f0-428f-a91a-1515f52c79f1 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:58.374750 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e9ab4-3df0-4ebb-9b0d-182506bc4a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:58.375437 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e9ab4-3df0-4ebb-9b0d-182506bc4a5f None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:58.375750 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e9ab4-3df0-4ebb-9b0d-182506bc4a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:58.376072 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e9ab4-3df0-4ebb-9b0d-182506bc4a5f None admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:58.376750 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e9ab4-3df0-4ebb-9b0d-182506bc4a5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:55:58.377086 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e9ab4-3df0-4ebb-9b0d-182506bc4a5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:58.378618 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e9ab4-3df0-4ebb-9b0d-182506bc4a5f None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:58.388009 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 477/954] 10.222.0.13 () {56 vars in 1099 bytes} [Sun Oct 23 19:55:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:55:58.394049 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99ccef-1666-43c7-8ccd-d2f1cf4359e2 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:58.499846 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:58.503340 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e99ccef-1666-43c7-8ccd-d2f1cf4359e2 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:58.539097 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2b27a-efe4-4248-b268-f8c57f9b02a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:58.539719 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2b27a-efe4-4248-b268-f8c57f9b02a5 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:58.539996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2b27a-efe4-4248-b268-f8c57f9b02a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:58.540266 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c2b27a-efe4-4248-b268-f8c57f9b02a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:58.552532 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2b27a-efe4-4248-b268-f8c57f9b02a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:58.554237 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c2b27a-efe4-4248-b268-f8c57f9b02a5 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:58.571777 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-e2c2b27a-efe4-4248-b268-f8c57f9b02a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:55:58.571777 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:55:58.571777 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:55:58.571777 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:55:58.571777 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:55:58.571777 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:55:58.571777 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:55:58.571777 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:55:58.571777 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:55:58.571777 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:55:58.571777 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:55:58.571777 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:55:58.571777 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:55:58.571777 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:55:58.571777 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:55:58.571777 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:55:58.577744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:55:58.577744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:55:58.577744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:55:58.577744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:55:58.577744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:55:58.577744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:55:58.577744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:55:58.577744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:55:58.577744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:55:58.577744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:55:58.577744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:55:58.577744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:55:58.577744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:55:58.577744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:55:58.577744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:55:58.577744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:55:58.577744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:55:58.578890 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:55:58.578890 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:55:58.578890 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:55:58.578890 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 478/955] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:55:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:55:58.579136 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e9ab4-3df0-4ebb-9b0d-182506bc4a5f None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:58.654654 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:58.693311 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e9ab4-3df0-4ebb-9b0d-182506bc4a5f None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:58.733870 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96314ea-0e81-48bb-99b8-4df17d49f782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:58.734587 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96314ea-0e81-48bb-99b8-4df17d49f782 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:58.734868 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96314ea-0e81-48bb-99b8-4df17d49f782 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:58.735140 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96314ea-0e81-48bb-99b8-4df17d49f782 None admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:58.735963 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96314ea-0e81-48bb-99b8-4df17d49f782 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:55:58.736315 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96314ea-0e81-48bb-99b8-4df17d49f782 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:58.737827 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96314ea-0e81-48bb-99b8-4df17d49f782 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:58.753086 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 478/956] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:55:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:55:58.759982 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2b27a-efe4-4248-b268-f8c57f9b02a5 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:58.872216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c2b27a-efe4-4248-b268-f8c57f9b02a5 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:58.911859 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6be602-41db-4f67-a057-321e2b75d7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:58.912335 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6be602-41db-4f67-a057-321e2b75d7bb None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:58.912611 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6be602-41db-4f67-a057-321e2b75d7bb None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:58.912882 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6be602-41db-4f67-a057-321e2b75d7bb None admin] PATH_INFO: `/v3/users/placement` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:58.921080 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6be602-41db-4f67-a057-321e2b75d7bb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:58.922540 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6be602-41db-4f67-a057-321e2b75d7bb None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:58.934225 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-4e6be602-41db-4f67-a057-321e2b75d7bb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 23 19:55:58.934225 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:55:58.934225 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:55:58.934225 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:55:58.934225 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:55:58.934225 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:55:58.934225 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:55:58.934225 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:55:58.934225 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:55:58.934225 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:55:58.934225 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:55:58.934225 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:55:58.934225 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 19:55:58.934225 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 19:55:58.934225 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 19:55:58.934225 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 19:55:58.935933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:55:58.935933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:55:58.935933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 19:55:58.935933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:55:58.935933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 19:55:58.935933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 19:55:58.935933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:55:58.935933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:55:58.935933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:55:58.935933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:55:58.935933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:55:58.935933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:55:58.935933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:55:58.935933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:55:58.935933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:55:58.935933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:55:58.935933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:55:58.935933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:55:58.936883 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:55:58.936883 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:55:58.936883 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 19:55:58.936883 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 19:55:58.936883 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 19:55:58.936883 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 19:55:58.936883 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 19:55:58.936883 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 19:55:58.936883 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 23 19:55:58.936883 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:55:58.936883 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 479/957] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:55:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:55:58.942075 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96314ea-0e81-48bb-99b8-4df17d49f782 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:59.056328 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96314ea-0e81-48bb-99b8-4df17d49f782 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:59.084696 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:59.093974 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ce033-0a47-406e-bce1-023656f41973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:59.094641 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ce033-0a47-406e-bce1-023656f41973 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:59.094922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ce033-0a47-406e-bce1-023656f41973 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:59.095197 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379ce033-0a47-406e-bce1-023656f41973 None admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:59.095956 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379ce033-0a47-406e-bce1-023656f41973 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:55:59.096262 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379ce033-0a47-406e-bce1-023656f41973 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:59.097713 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379ce033-0a47-406e-bce1-023656f41973 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:59.117525 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 479/958] 10.222.0.13 () {56 vars in 1143 bytes} [Sun Oct 23 19:55:58 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:55:59.124013 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6be602-41db-4f67-a057-321e2b75d7bb None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:59.140480 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:59.234692 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6be602-41db-4f67-a057-321e2b75d7bb None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:59.270929 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84fb41f-f332-4fb1-8627-8a098325d541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:59.271554 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fb41f-f332-4fb1-8627-8a098325d541 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:59.271834 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fb41f-f332-4fb1-8627-8a098325d541 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:59.272108 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84fb41f-f332-4fb1-8627-8a098325d541 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:59.343293 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84fb41f-f332-4fb1-8627-8a098325d541 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:59.344765 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84fb41f-f332-4fb1-8627-8a098325d541 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:59.359754 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:59.451941 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 480/959] 10.222.0.13 () {58 vars in 1282 bytes} [Sun Oct 23 19:55:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 23 19:55:59.458040 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ce033-0a47-406e-bce1-023656f41973 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:59.559014 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379ce033-0a47-406e-bce1-023656f41973 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:59.593898 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b0fc5-7736-4d74-8c92-2394de9b73e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:59.594635 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b0fc5-7736-4d74-8c92-2394de9b73e3 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:59.594920 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b0fc5-7736-4d74-8c92-2394de9b73e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:59.595198 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190b0fc5-7736-4d74-8c92-2394de9b73e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:59.605943 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190b0fc5-7736-4d74-8c92-2394de9b73e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:59.608001 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190b0fc5-7736-4d74-8c92-2394de9b73e3 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:59.623020 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-190b0fc5-7736-4d74-8c92-2394de9b73e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:55:59.623020 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:55:59.623020 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:55:59.623020 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:55:59.623020 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:55:59.623020 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:55:59.623020 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:55:59.623020 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:55:59.623020 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:55:59.623020 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:55:59.623020 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:55:59.623020 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:55:59.623020 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 19:55:59.623020 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 19:55:59.623020 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 19:55:59.623020 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 19:55:59.623971 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:55:59.623971 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:55:59.623971 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:55:59.623971 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:55:59.623971 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:55:59.623971 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:55:59.623971 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:55:59.623971 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:55:59.623971 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:55:59.623971 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:55:59.623971 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:55:59.623971 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:55:59.623971 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:55:59.623971 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:55:59.623971 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:55:59.623971 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:55:59.623971 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 19:55:59.624845 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 19:55:59.624845 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 19:55:59.624845 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:55:59.624845 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 480/960] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 19:55:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:55:59.630141 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84fb41f-f332-4fb1-8627-8a098325d541 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:59.725242 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84fb41f-f332-4fb1-8627-8a098325d541 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:59.738348 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:59.762108 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb15f7f-6e2f-4ad1-9cc2-e199827ce226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:59.762826 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb15f7f-6e2f-4ad1-9cc2-e199827ce226 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:59.763139 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb15f7f-6e2f-4ad1-9cc2-e199827ce226 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:59.763519 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb15f7f-6e2f-4ad1-9cc2-e199827ce226 None admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:59.764269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb15f7f-6e2f-4ad1-9cc2-e199827ce226 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:55:59.764579 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb15f7f-6e2f-4ad1-9cc2-e199827ce226 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:59.766084 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb15f7f-6e2f-4ad1-9cc2-e199827ce226 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:59.779166 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 481/961] 10.222.0.13 () {56 vars in 1109 bytes} [Sun Oct 23 19:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:55:59.785328 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b0fc5-7736-4d74-8c92-2394de9b73e3 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:55:59.820390 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:55:59.883904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190b0fc5-7736-4d74-8c92-2394de9b73e3 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:55:59.916006 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ad32a-1b44-4a65-b60a-faf4aae932b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:55:59.916618 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ad32a-1b44-4a65-b60a-faf4aae932b7 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:55:59.916897 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ad32a-1b44-4a65-b60a-faf4aae932b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:55:59.917168 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0ad32a-1b44-4a65-b60a-faf4aae932b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:55:59.924112 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ad32a-1b44-4a65-b60a-faf4aae932b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:55:59.925521 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0ad32a-1b44-4a65-b60a-faf4aae932b7 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:55:59.936461 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application [None req-6e0ad32a-1b44-4a65-b60a-faf4aae932b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:55:59.936461 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:55:59.936461 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:55:59.936461 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:55:59.936461 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:55:59.936461 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:55:59.936461 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:55:59.936461 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:55:59.936461 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:55:59.936461 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:55:59.936461 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:55:59.936461 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:55:59.936461 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 19:55:59.936461 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 19:55:59.936461 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 19:55:59.937493 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 19:55:59.937493 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:55:59.937493 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:55:59.937493 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:55:59.937493 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:55:59.937493 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:55:59.937493 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:55:59.937493 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:55:59.937493 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application with Lock( Oct 23 19:55:59.937493 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:55:59.937493 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:55:59.937493 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:55:59.937493 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:55:59.937493 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:55:59.937493 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:55:59.937493 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:55:59.937493 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:55:59.938645 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 19:55:59.938645 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 19:55:59.938645 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 19:55:59.938645 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 19:55:59.938645 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 19:55:59.938645 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 19:55:59.938645 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 19:55:59.938645 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: ERROR keystone.server.flask.application Oct 23 19:55:59.938645 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 481/962] 10.222.0.13 () {56 vars in 1111 bytes} [Sun Oct 23 19:55:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 19:55:59.943536 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb15f7f-6e2f-4ad1-9cc2-e199827ce226 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:56:00.040630 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb15f7f-6e2f-4ad1-9cc2-e199827ce226 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:56:00.074769 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00db2a0-a6a0-4290-8950-26a9f56ffab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:56:00.075306 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00db2a0-a6a0-4290-8950-26a9f56ffab4 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:56:00.075629 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00db2a0-a6a0-4290-8950-26a9f56ffab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:56:00.075904 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00db2a0-a6a0-4290-8950-26a9f56ffab4 None admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:56:00.076593 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00db2a0-a6a0-4290-8950-26a9f56ffab4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:56:00.076902 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00db2a0-a6a0-4290-8950-26a9f56ffab4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:56:00.078343 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00db2a0-a6a0-4290-8950-26a9f56ffab4 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:56:00.089883 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 482/963] 10.222.0.13 () {56 vars in 1148 bytes} [Sun Oct 23 19:55:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:56:00.095898 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ad32a-1b44-4a65-b60a-faf4aae932b7 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:56:00.186845 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:56:00.193092 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0ad32a-1b44-4a65-b60a-faf4aae932b7 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:56:00.224834 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21744232-35c0-4709-bab5-ea96c3e23577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:56:00.225421 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21744232-35c0-4709-bab5-ea96c3e23577 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:56:00.225700 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21744232-35c0-4709-bab5-ea96c3e23577 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:56:00.225970 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21744232-35c0-4709-bab5-ea96c3e23577 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:56:00.226716 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21744232-35c0-4709-bab5-ea96c3e23577 None admin] RBAC: Adding query filter params (role.id=742dfaf63ef74207a2c9213411513925, scope.project.id=d40947427f5f43e7b6c82e3605737767, user.id=312489c49a304ca9aacebd9843a4b65f) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:56:00.227026 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21744232-35c0-4709-bab5-ea96c3e23577 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:56:00.228418 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21744232-35c0-4709-bab5-ea96c3e23577 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:56:00.234520 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 482/964] 10.222.0.13 () {56 vars in 1374 bytes} [Sun Oct 23 19:56:00 2022] GET /identity/v3/role_assignments?user.id=312489c49a304ca9aacebd9843a4b65f&scope.project.id=d40947427f5f43e7b6c82e3605737767&role.id=742dfaf63ef74207a2c9213411513925 => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:56:02.359798 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfdaa1f-4ba5-4761-8f03-a30d78d862fc None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:56:02.360305 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfdaa1f-4ba5-4761-8f03-a30d78d862fc None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:56:02.360428 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfdaa1f-4ba5-4761-8f03-a30d78d862fc None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:56:02.361702 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 483/965] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:56:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:56:02.367126 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478203d-bd5e-4efc-ac50-5a157d01258e None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:56:02.367554 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478203d-bd5e-4efc-ac50-5a157d01258e None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:56:02.367676 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f478203d-bd5e-4efc-ac50-5a157d01258e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:56:02.385909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-f478203d-bd5e-4efc-ac50-5a157d01258e None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:56:02.387108 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f478203d-bd5e-4efc-ac50-5a157d01258e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:56:02.450753 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 483/966] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:56:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:56:02.713926 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfdaa1f-4ba5-4761-8f03-a30d78d862fc None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:56:02.747426 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:56:02.832964 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfdaa1f-4ba5-4761-8f03-a30d78d862fc None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:56:02.865323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04046d26-a85a-48ba-9ef3-0f8be1065610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:56:02.865934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04046d26-a85a-48ba-9ef3-0f8be1065610 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:56:02.866208 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04046d26-a85a-48ba-9ef3-0f8be1065610 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:56:02.866543 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04046d26-a85a-48ba-9ef3-0f8be1065610 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:56:02.867120 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04046d26-a85a-48ba-9ef3-0f8be1065610 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:56:02.868542 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04046d26-a85a-48ba-9ef3-0f8be1065610 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:56:02.877659 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-04046d26-a85a-48ba-9ef3-0f8be1065610 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 23 19:56:02.877659 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:56:02.877659 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 19:56:02.877659 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 19:56:02.877659 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 19:56:02.877659 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 19:56:02.877659 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 19:56:02.877659 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:56:02.877659 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 23 19:56:02.877659 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:56:02.877659 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 19:56:02.877659 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:56:02.877659 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:56:02.877659 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:56:02.877659 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:56:02.877659 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:56:02.877659 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:56:02.878876 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:56:02.878876 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:56:02.878876 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:56:02.878876 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:56:02.878876 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:56:02.878876 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:56:02.878876 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 19:56:02.878876 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 19:56:02.878876 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 19:56:02.878876 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 19:56:02.878876 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:56:02.878876 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:56:02.878876 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:56:02.878876 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:56:02.878876 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:56:02.878876 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:56:02.879744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:56:02.879744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:56:02.879744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:56:02.879744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:56:02.879744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:56:02.879744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:56:02.879744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:56:02.879744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:56:02.879744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:56:02.879744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:56:02.879744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 19:56:02.879744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:56:02.879744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 23 19:56:02.879744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:56:02.879744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 484/967] 10.222.0.13 () {56 vars in 1117 bytes} [Sun Oct 23 19:56:02 2022] GET /identity/v3/services/placement => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:56:02.884929 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478203d-bd5e-4efc-ac50-5a157d01258e None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:56:02.971576 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:56:02.981119 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f478203d-bd5e-4efc-ac50-5a157d01258e None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:56:03.014426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518cbf2e-1219-4246-9990-70560a69e49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:56:03.015053 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cbf2e-1219-4246-9990-70560a69e49a None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:56:03.015326 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cbf2e-1219-4246-9990-70560a69e49a None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:56:03.015591 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518cbf2e-1219-4246-9990-70560a69e49a None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:56:03.016209 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518cbf2e-1219-4246-9990-70560a69e49a None admin] RBAC: Adding query filter params (name=placement) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:56:03.016512 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518cbf2e-1219-4246-9990-70560a69e49a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:56:03.017857 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518cbf2e-1219-4246-9990-70560a69e49a None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:56:03.022768 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 484/968] 10.222.0.13 () {56 vars in 1116 bytes} [Sun Oct 23 19:56:02 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:56:03.028359 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04046d26-a85a-48ba-9ef3-0f8be1065610 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:56:03.120553 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:56:03.127010 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04046d26-a85a-48ba-9ef3-0f8be1065610 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:56:03.158714 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86583edc-ac19-42d3-9791-8d75193a2c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:56:03.159299 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86583edc-ac19-42d3-9791-8d75193a2c72 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:56:03.159572 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86583edc-ac19-42d3-9791-8d75193a2c72 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:56:03.159840 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86583edc-ac19-42d3-9791-8d75193a2c72 None admin] PATH_INFO: `/v3/services` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:56:03.160456 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86583edc-ac19-42d3-9791-8d75193a2c72 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:56:03.160751 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86583edc-ac19-42d3-9791-8d75193a2c72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:56:03.162093 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86583edc-ac19-42d3-9791-8d75193a2c72 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:56:03.166370 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 485/969] 10.222.0.13 () {56 vars in 1116 bytes} [Sun Oct 23 19:56:03 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:56:05.278798 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef299057-ecab-47f6-b758-0f8864010f7f None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:56:05.279344 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef299057-ecab-47f6-b758-0f8864010f7f None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:56:05.279344 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef299057-ecab-47f6-b758-0f8864010f7f None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:56:05.280602 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 485/970] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:56:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:56:05.285940 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6d9ee-734c-4125-876a-9a149ae95aa8 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:56:05.286415 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6d9ee-734c-4125-876a-9a149ae95aa8 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:56:05.286514 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a6d9ee-734c-4125-876a-9a149ae95aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:56:05.304629 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-38a6d9ee-734c-4125-876a-9a149ae95aa8 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:56:05.305787 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-38a6d9ee-734c-4125-876a-9a149ae95aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:56:05.369387 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 486/971] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:56:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:56:05.633826 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef299057-ecab-47f6-b758-0f8864010f7f None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:56:05.731717 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef299057-ecab-47f6-b758-0f8864010f7f None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:56:05.767439 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c42017-7337-4935-82e2-72acd0e0114f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:56:05.768079 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c42017-7337-4935-82e2-72acd0e0114f None admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:56:05.768398 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c42017-7337-4935-82e2-72acd0e0114f None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:56:05.768693 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c42017-7337-4935-82e2-72acd0e0114f None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:56:05.769364 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c42017-7337-4935-82e2-72acd0e0114f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:56:05.770871 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c42017-7337-4935-82e2-72acd0e0114f None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:56:05.786553 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 486/972] 10.222.0.13 () {60 vars in 1141 bytes} [Sun Oct 23 19:56:05 2022] POST /identity/v3/services => generated 246 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:56:07.920528 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00c7c7-68b2-4ff4-89fb-7428b6495645 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:56:07.921050 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00c7c7-68b2-4ff4-89fb-7428b6495645 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:56:07.921050 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00c7c7-68b2-4ff4-89fb-7428b6495645 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:56:07.922455 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 487/973] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:56:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:56:07.927624 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c49f9f-1c24-42d8-9e84-662602e4f33f None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:56:07.927995 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c49f9f-1c24-42d8-9e84-662602e4f33f None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:56:07.928183 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c49f9f-1c24-42d8-9e84-662602e4f33f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:56:07.946987 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-e1c49f9f-1c24-42d8-9e84-662602e4f33f None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:56:07.948206 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e1c49f9f-1c24-42d8-9e84-662602e4f33f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:56:07.957815 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:56:08.021844 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 487/974] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:56:07 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:56:08.288319 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd00c7c7-68b2-4ff4-89fb-7428b6495645 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:56:08.375398 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:56:08.385728 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd00c7c7-68b2-4ff4-89fb-7428b6495645 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:56:08.418419 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b72074-b720-4ac2-9b2b-7d83c2ac6263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:56:08.419029 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b72074-b720-4ac2-9b2b-7d83c2ac6263 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:56:08.419328 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b72074-b720-4ac2-9b2b-7d83c2ac6263 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:56:08.419598 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b72074-b720-4ac2-9b2b-7d83c2ac6263 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:56:08.420172 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b72074-b720-4ac2-9b2b-7d83c2ac6263 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:56:08.421512 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b72074-b720-4ac2-9b2b-7d83c2ac6263 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:56:08.429958 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-b3b72074-b720-4ac2-9b2b-7d83c2ac6263 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 23 19:56:08.429958 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:56:08.429958 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 19:56:08.429958 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 19:56:08.429958 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 19:56:08.429958 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 19:56:08.429958 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 19:56:08.429958 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:56:08.429958 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 23 19:56:08.429958 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:56:08.429958 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 19:56:08.429958 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:56:08.429958 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:56:08.429958 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:56:08.429958 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:56:08.429958 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:56:08.429958 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:56:08.431316 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:56:08.431316 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:56:08.431316 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:56:08.431316 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:56:08.431316 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:56:08.431316 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:56:08.431316 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 19:56:08.431316 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 19:56:08.431316 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 19:56:08.431316 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 19:56:08.431316 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:56:08.431316 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:56:08.431316 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:56:08.431316 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:56:08.431316 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:56:08.431316 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:56:08.432544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:56:08.432544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:56:08.432544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:56:08.432544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:56:08.432544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:56:08.432544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:56:08.432544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:56:08.432544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:56:08.432544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:56:08.432544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:56:08.432544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 19:56:08.432544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:56:08.432544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 23 19:56:08.432544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:56:08.432544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 488/975] 10.222.0.13 () {56 vars in 1117 bytes} [Sun Oct 23 19:56:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:56:08.436953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c49f9f-1c24-42d8-9e84-662602e4f33f None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:56:08.533375 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c49f9f-1c24-42d8-9e84-662602e4f33f None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:56:08.565402 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb93791-0aa3-42c6-911d-d8fa3d4d3782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:56:08.565940 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb93791-0aa3-42c6-911d-d8fa3d4d3782 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:56:08.566219 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb93791-0aa3-42c6-911d-d8fa3d4d3782 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:56:08.566767 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb93791-0aa3-42c6-911d-d8fa3d4d3782 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:56:08.567390 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb93791-0aa3-42c6-911d-d8fa3d4d3782 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:56:08.567689 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb93791-0aa3-42c6-911d-d8fa3d4d3782 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:56:08.569025 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb93791-0aa3-42c6-911d-d8fa3d4d3782 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:56:08.573939 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 488/976] 10.222.0.13 () {56 vars in 1116 bytes} [Sun Oct 23 19:56:08 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:56:08.580127 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b72074-b720-4ac2-9b2b-7d83c2ac6263 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:56:08.677211 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b72074-b720-4ac2-9b2b-7d83c2ac6263 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:56:08.710265 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20211917-9da4-4cb8-8674-3ca2e2b2219f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:56:08.710875 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20211917-9da4-4cb8-8674-3ca2e2b2219f None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:56:08.711156 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20211917-9da4-4cb8-8674-3ca2e2b2219f None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:56:08.711430 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20211917-9da4-4cb8-8674-3ca2e2b2219f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:56:08.712125 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20211917-9da4-4cb8-8674-3ca2e2b2219f None admin] RBAC: Adding query filter params (interface=public, service_id=b3ca14c186f54c00b6f97e8f742a7664, region_id=RegionOne) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:56:08.712433 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20211917-9da4-4cb8-8674-3ca2e2b2219f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:56:08.713785 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20211917-9da4-4cb8-8674-3ca2e2b2219f None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:56:08.720101 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 489/977] 10.222.0.13 () {56 vars in 1251 bytes} [Sun Oct 23 19:56:08 2022] GET /identity/v3/endpoints?service_id=b3ca14c186f54c00b6f97e8f742a7664&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:56:08.725621 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb93791-0aa3-42c6-911d-d8fa3d4d3782 None admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:56:08.729327 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:56:08.843741 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb93791-0aa3-42c6-911d-d8fa3d4d3782 None admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:56:08.875999 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb924fd5-f4d5-40b6-861a-a68a63deae70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:56:08.876595 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb924fd5-f4d5-40b6-861a-a68a63deae70 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:56:08.876870 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb924fd5-f4d5-40b6-861a-a68a63deae70 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:56:08.877136 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb924fd5-f4d5-40b6-861a-a68a63deae70 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:56:08.877727 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb924fd5-f4d5-40b6-861a-a68a63deae70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:56:08.879086 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb924fd5-f4d5-40b6-861a-a68a63deae70 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:56:08.885597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 489/978] 10.222.0.13 () {56 vars in 1087 bytes} [Sun Oct 23 19:56:08 2022] GET /identity/v3/services => generated 2191 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 19:56:10.991224 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164db948-a900-4408-a78e-ca4276c2498a None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:56:10.991792 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164db948-a900-4408-a78e-ca4276c2498a None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:56:10.991792 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164db948-a900-4408-a78e-ca4276c2498a None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:56:10.993105 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 490/979] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:56:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:56:10.998267 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e5fe3-4575-492d-87a2-265d2f116854 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:56:10.998681 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e5fe3-4575-492d-87a2-265d2f116854 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:56:10.998813 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e5fe3-4575-492d-87a2-265d2f116854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:56:11.016738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-648e5fe3-4575-492d-87a2-265d2f116854 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:56:11.017917 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-648e5fe3-4575-492d-87a2-265d2f116854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:56:11.077755 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:56:11.083562 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 490/980] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:56:10 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 19:56:11.350983 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164db948-a900-4408-a78e-ca4276c2498a None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:56:11.370381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:56:11.449269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164db948-a900-4408-a78e-ca4276c2498a None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:56:11.481185 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9040d59-0ab0-4045-9b38-a20c3cbdc0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:56:11.482331 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9040d59-0ab0-4045-9b38-a20c3cbdc0f4 None admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:56:11.482610 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9040d59-0ab0-4045-9b38-a20c3cbdc0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:56:11.482878 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9040d59-0ab0-4045-9b38-a20c3cbdc0f4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:56:11.483448 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9040d59-0ab0-4045-9b38-a20c3cbdc0f4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:56:11.484793 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9040d59-0ab0-4045-9b38-a20c3cbdc0f4 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:56:11.493381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application [None req-b9040d59-0ab0-4045-9b38-a20c3cbdc0f4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 23 19:56:11.493381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:56:11.493381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 19:56:11.493381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 19:56:11.493381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 19:56:11.493381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 19:56:11.493381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 19:56:11.493381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:56:11.493381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 23 19:56:11.493381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:56:11.493381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 19:56:11.493381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:56:11.493381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 19:56:11.493381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 19:56:11.493381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 19:56:11.493381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 19:56:11.493381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 19:56:11.494434 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 19:56:11.494434 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 19:56:11.494434 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 19:56:11.494434 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 19:56:11.494434 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 19:56:11.494434 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 19:56:11.494434 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 19:56:11.494434 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 19:56:11.494434 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 19:56:11.494434 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 19:56:11.494434 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 19:56:11.494434 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 19:56:11.494434 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 19:56:11.494434 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 19:56:11.494434 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 19:56:11.494434 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 19:56:11.495302 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 19:56:11.495302 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application with Lock( Oct 23 19:56:11.495302 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 19:56:11.495302 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self._enter() Oct 23 19:56:11.495302 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 19:56:11.495302 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 19:56:11.495302 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 19:56:11.495302 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application return self.creator() Oct 23 19:56:11.495302 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 19:56:11.495302 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application created_value = creator( Oct 23 19:56:11.495302 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 19:56:11.495302 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 19:56:11.495302 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 23 19:56:11.495302 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: ERROR keystone.server.flask.application Oct 23 19:56:11.495302 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 491/981] 10.222.0.13 () {56 vars in 1117 bytes} [Sun Oct 23 19:56:11 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 19:56:11.500900 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648e5fe3-4575-492d-87a2-265d2f116854 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:56:11.595992 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648e5fe3-4575-492d-87a2-265d2f116854 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:56:11.629150 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aecf60e-a5c7-4afb-adfc-a5f66171f294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:56:11.629683 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecf60e-a5c7-4afb-adfc-a5f66171f294 None admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:56:11.629959 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecf60e-a5c7-4afb-adfc-a5f66171f294 None admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:56:11.630232 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecf60e-a5c7-4afb-adfc-a5f66171f294 None admin] PATH_INFO: `/v3/services` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:56:11.630875 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aecf60e-a5c7-4afb-adfc-a5f66171f294 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 19:56:11.631175 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aecf60e-a5c7-4afb-adfc-a5f66171f294 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:56:11.632513 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aecf60e-a5c7-4afb-adfc-a5f66171f294 None admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:56:11.637426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 491/982] 10.222.0.13 () {56 vars in 1116 bytes} [Sun Oct 23 19:56:11 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 19:56:11.642909 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9040d59-0ab0-4045-9b38-a20c3cbdc0f4 None admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:56:11.721909 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:56:11.740111 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9040d59-0ab0-4045-9b38-a20c3cbdc0f4 None admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:56:11.779703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529aae8b-459f-486b-8e5f-59a13da74de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:56:11.780490 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529aae8b-459f-486b-8e5f-59a13da74de0 None admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:56:11.780952 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529aae8b-459f-486b-8e5f-59a13da74de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:56:11.781407 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529aae8b-459f-486b-8e5f-59a13da74de0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:56:11.782210 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529aae8b-459f-486b-8e5f-59a13da74de0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:56:11.783666 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529aae8b-459f-486b-8e5f-59a13da74de0 None admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:56:11.806645 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 492/983] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 19:56:11 2022] POST /identity/v3/endpoints => generated 331 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 19:56:21.634495 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=69 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:56:21.801885 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:57:14.969059 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49153202-f5dd-4247-b9a8-7b79bc19f6e2 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:14.970095 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49153202-f5dd-4247-b9a8-7b79bc19f6e2 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:14.970632 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49153202-f5dd-4247-b9a8-7b79bc19f6e2 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:14.975836 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 492/984] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:57:14 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:57:14.977583 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a42c0-c02a-48b6-9fee-a8bdb7713868 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:14.977997 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a42c0-c02a-48b6-9fee-a8bdb7713868 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:14.978198 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706a42c0-c02a-48b6-9fee-a8bdb7713868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:15.100435 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-706a42c0-c02a-48b6-9fee-a8bdb7713868 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:57:15.105608 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-706a42c0-c02a-48b6-9fee-a8bdb7713868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:15.306887 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 493/985] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:57:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:57:15.596213 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892ce2a-c816-4484-88b6-f4c9af9e961a None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:15.596581 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892ce2a-c816-4484-88b6-f4c9af9e961a None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:15.596982 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892ce2a-c816-4484-88b6-f4c9af9e961a None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:15.598442 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 493/986] 10.222.0.13 () {54 vars in 964 bytes} [Sun Oct 23 19:57:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:57:15.604418 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131ff13-8966-4ffe-b039-22ba112f7f10 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:15.604967 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131ff13-8966-4ffe-b039-22ba112f7f10 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:15.605463 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d131ff13-8966-4ffe-b039-22ba112f7f10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:15.687302 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-d131ff13-8966-4ffe-b039-22ba112f7f10 None None] MFA Rules not processed for user `ba8ff3413bc549d29486d1c724a1f383`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:57:15.688631 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-d131ff13-8966-4ffe-b039-22ba112f7f10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:15.711323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:57:15.772207 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 494/987] 10.222.0.13 () {58 vars in 1061 bytes} [Sun Oct 23 19:57:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:57:15.780987 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3892ce2a-c816-4484-88b6-f4c9af9e961a None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:57:15.782766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3892ce2a-c816-4484-88b6-f4c9af9e961a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:15.912013 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3892ce2a-c816-4484-88b6-f4c9af9e961a None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:57:16.002052 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4155431d-e987-4345-ae11-b3478a4c7b59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:57:16.002740 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155431d-e987-4345-ae11-b3478a4c7b59 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:16.003011 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155431d-e987-4345-ae11-b3478a4c7b59 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:16.003218 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4155431d-e987-4345-ae11-b3478a4c7b59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:16.004845 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4155431d-e987-4345-ae11-b3478a4c7b59 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:16.067840 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:57:16.127046 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4155431d-e987-4345-ae11-b3478a4c7b59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:57:16.128604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4155431d-e987-4345-ae11-b3478a4c7b59 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:57:16.148555 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 494/988] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:57:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:57:18.639625 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f597185b-0973-4870-a660-0b4e97ae8505 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:18.640139 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f597185b-0973-4870-a660-0b4e97ae8505 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:18.640229 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f597185b-0973-4870-a660-0b4e97ae8505 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:18.641532 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 495/989] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:57:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:57:18.646795 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a9cbc-e7df-41fa-8be1-582a407242d5 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:18.647209 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a9cbc-e7df-41fa-8be1-582a407242d5 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:18.647368 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a9cbc-e7df-41fa-8be1-582a407242d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:18.732490 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-ea8a9cbc-e7df-41fa-8be1-582a407242d5 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:57:18.733686 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ea8a9cbc-e7df-41fa-8be1-582a407242d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:18.829867 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 495/990] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:57:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:57:19.113256 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f597185b-0973-4870-a660-0b4e97ae8505 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:57:19.114815 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-f597185b-0973-4870-a660-0b4e97ae8505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:19.216877 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f597185b-0973-4870-a660-0b4e97ae8505 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:57:19.286323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:57:19.288806 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944a10d5-c34f-4427-b40f-4e7876b5058b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:57:19.289605 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a10d5-c34f-4427-b40f-4e7876b5058b service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:19.290049 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a10d5-c34f-4427-b40f-4e7876b5058b service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:19.290522 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944a10d5-c34f-4427-b40f-4e7876b5058b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:19.292131 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-944a10d5-c34f-4427-b40f-4e7876b5058b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:19.398518 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a10d5-c34f-4427-b40f-4e7876b5058b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:57:19.399945 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944a10d5-c34f-4427-b40f-4e7876b5058b service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:57:19.417333 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 496/991] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:57:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:57:21.838307 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07cad6b-72a3-4d35-b03a-fb94eeef69fa None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:21.838842 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07cad6b-72a3-4d35-b03a-fb94eeef69fa None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:21.838842 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07cad6b-72a3-4d35-b03a-fb94eeef69fa None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:21.840282 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 496/992] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:57:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:57:21.845547 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323035c-0cd3-4798-95b2-7e2d847b635a None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:21.845951 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323035c-0cd3-4798-95b2-7e2d847b635a None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:21.846185 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323035c-0cd3-4798-95b2-7e2d847b635a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:21.910308 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-3323035c-0cd3-4798-95b2-7e2d847b635a None None] MFA Rules not processed for user `d158c81dd66f4f0181f82eb165a7b8a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:57:21.911484 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-3323035c-0cd3-4798-95b2-7e2d847b635a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:21.962499 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:57:22.006951 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 497/993] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:57:21 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:57:22.292194 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07cad6b-72a3-4d35-b03a-fb94eeef69fa None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:57:22.294327 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c07cad6b-72a3-4d35-b03a-fb94eeef69fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:22.303953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:57:22.396294 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07cad6b-72a3-4d35-b03a-fb94eeef69fa None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:57:22.469605 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbb3dc3-4ca9-4675-8cc9-3307c0b6ce15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:57:22.470198 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb3dc3-4ca9-4675-8cc9-3307c0b6ce15 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:22.470499 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb3dc3-4ca9-4675-8cc9-3307c0b6ce15 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:22.470764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbb3dc3-4ca9-4675-8cc9-3307c0b6ce15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:22.472320 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3fbb3dc3-4ca9-4675-8cc9-3307c0b6ce15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:22.531872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:57:22.578544 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb3dc3-4ca9-4675-8cc9-3307c0b6ce15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:57:22.579959 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbb3dc3-4ca9-4675-8cc9-3307c0b6ce15 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:57:22.597786 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 497/994] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:57:22 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:57:25.231614 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd584e-10b0-4f8a-ba51-5e911f4cf81a None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:25.232175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd584e-10b0-4f8a-ba51-5e911f4cf81a None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:25.232175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fd584e-10b0-4f8a-ba51-5e911f4cf81a None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:25.233500 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 498/995] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:57:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:57:25.238830 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d30d64-eaca-4cb9-99d7-9171a51c0864 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:25.239214 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d30d64-eaca-4cb9-99d7-9171a51c0864 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:25.239380 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d30d64-eaca-4cb9-99d7-9171a51c0864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:25.303852 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-f8d30d64-eaca-4cb9-99d7-9171a51c0864 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:57:25.304969 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f8d30d64-eaca-4cb9-99d7-9171a51c0864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:25.399659 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 498/996] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:57:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:57:25.681768 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd584e-10b0-4f8a-ba51-5e911f4cf81a None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:57:25.683329 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-a7fd584e-10b0-4f8a-ba51-5e911f4cf81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:25.786524 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fd584e-10b0-4f8a-ba51-5e911f4cf81a None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:57:25.858419 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab855154-2f23-4616-85fe-c80d968ff29e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:57:25.859011 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab855154-2f23-4616-85fe-c80d968ff29e service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:25.859285 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab855154-2f23-4616-85fe-c80d968ff29e service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:25.859549 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab855154-2f23-4616-85fe-c80d968ff29e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:25.861124 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-ab855154-2f23-4616-85fe-c80d968ff29e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:25.867799 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:57:25.967372 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab855154-2f23-4616-85fe-c80d968ff29e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:57:25.968713 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab855154-2f23-4616-85fe-c80d968ff29e service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:57:25.985932 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 499/997] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:57:25 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:57:28.765184 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa43c54-6eac-42b1-bec6-48d3a8e4f18b None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:28.765765 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa43c54-6eac-42b1-bec6-48d3a8e4f18b None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:28.765765 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa43c54-6eac-42b1-bec6-48d3a8e4f18b None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:28.767391 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 499/998] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:57:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:57:28.772991 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50250c5-caea-453a-83d2-b1c4cc24d864 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:28.773489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50250c5-caea-453a-83d2-b1c4cc24d864 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:28.773738 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50250c5-caea-453a-83d2-b1c4cc24d864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:28.846493 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-e50250c5-caea-453a-83d2-b1c4cc24d864 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:57:28.847715 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-e50250c5-caea-453a-83d2-b1c4cc24d864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:28.909324 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:57:28.950950 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 500/999] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:57:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:57:29.229103 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa43c54-6eac-42b1-bec6-48d3a8e4f18b None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:57:29.231293 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ffa43c54-6eac-42b1-bec6-48d3a8e4f18b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:29.250900 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:57:29.334306 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa43c54-6eac-42b1-bec6-48d3a8e4f18b None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:57:29.406663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f68f68c-af6e-45d2-842b-1ee590b27e86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:57:29.407273 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f68f68c-af6e-45d2-842b-1ee590b27e86 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:29.407544 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f68f68c-af6e-45d2-842b-1ee590b27e86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:29.407811 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f68f68c-af6e-45d2-842b-1ee590b27e86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:29.409382 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7f68f68c-af6e-45d2-842b-1ee590b27e86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:29.481759 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:57:29.521092 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f68f68c-af6e-45d2-842b-1ee590b27e86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:57:29.522597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f68f68c-af6e-45d2-842b-1ee590b27e86 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:57:29.540257 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 500/1000] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:57:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:57:31.737314 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4a2fe-addd-4a0f-bbfc-b600198024b2 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:31.737866 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4a2fe-addd-4a0f-bbfc-b600198024b2 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:31.737866 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb4a2fe-addd-4a0f-bbfc-b600198024b2 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:31.739231 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 501/1001] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:57:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:57:31.744339 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac974c-9c68-4dca-9b98-746ddd199f5b None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:31.744745 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac974c-9c68-4dca-9b98-746ddd199f5b None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:31.744882 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac974c-9c68-4dca-9b98-746ddd199f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:31.811198 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-f2ac974c-9c68-4dca-9b98-746ddd199f5b None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:57:31.812390 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f2ac974c-9c68-4dca-9b98-746ddd199f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:31.946344 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 501/1002] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:57:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:57:32.234702 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5fb91-4925-4fa8-a00f-17ef98858772 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:32.235106 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5fb91-4925-4fa8-a00f-17ef98858772 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:32.235454 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada5fb91-4925-4fa8-a00f-17ef98858772 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:32.236935 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 502/1003] 10.222.0.13 () {54 vars in 964 bytes} [Sun Oct 23 19:57:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:57:32.243009 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75abbf5f-7063-4edd-a13e-db9a8a0796e8 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:32.243361 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75abbf5f-7063-4edd-a13e-db9a8a0796e8 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:32.243640 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75abbf5f-7063-4edd-a13e-db9a8a0796e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:32.276183 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:57:32.323542 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-75abbf5f-7063-4edd-a13e-db9a8a0796e8 None None] MFA Rules not processed for user `ba8ff3413bc549d29486d1c724a1f383`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:57:32.324922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-75abbf5f-7063-4edd-a13e-db9a8a0796e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:32.403820 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 502/1004] 10.222.0.13 () {58 vars in 1061 bytes} [Sun Oct 23 19:57:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 19:57:32.412662 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5fb91-4925-4fa8-a00f-17ef98858772 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:57:32.414169 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-ada5fb91-4925-4fa8-a00f-17ef98858772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:32.517806 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada5fb91-4925-4fa8-a00f-17ef98858772 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:57:32.591639 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ebe90-dc3c-4b76-b260-d08f6f84e727 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:57:32.592236 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ebe90-dc3c-4b76-b260-d08f6f84e727 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:32.592507 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ebe90-dc3c-4b76-b260-d08f6f84e727 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:32.592774 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ebe90-dc3c-4b76-b260-d08f6f84e727 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:32.594450 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-de5ebe90-dc3c-4b76-b260-d08f6f84e727 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:32.611051 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:57:32.702353 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ebe90-dc3c-4b76-b260-d08f6f84e727 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:57:32.704239 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ebe90-dc3c-4b76-b260-d08f6f84e727 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:57:32.722750 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 503/1005] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:57:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:57:35.168042 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725429a-3623-4946-8865-6bb2f5d02d47 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:35.168495 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725429a-3623-4946-8865-6bb2f5d02d47 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:35.168684 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725429a-3623-4946-8865-6bb2f5d02d47 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:35.169983 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 503/1006] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:57:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:57:35.175419 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6da94-114a-402f-8e51-5f0dff1e0798 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:35.175888 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6da94-114a-402f-8e51-5f0dff1e0798 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:35.175978 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6da94-114a-402f-8e51-5f0dff1e0798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:35.246031 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-63f6da94-114a-402f-8e51-5f0dff1e0798 None None] MFA Rules not processed for user `d158c81dd66f4f0181f82eb165a7b8a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:57:35.247800 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-63f6da94-114a-402f-8e51-5f0dff1e0798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:35.313931 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:57:35.345845 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 504/1007] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:57:35 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:57:35.627463 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725429a-3623-4946-8865-6bb2f5d02d47 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:57:35.628987 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9725429a-3623-4946-8865-6bb2f5d02d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:35.735628 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725429a-3623-4946-8865-6bb2f5d02d47 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:57:35.757039 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:57:35.814739 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cabb304-75e8-41d2-b9a4-cd411f5a02d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:57:35.815349 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cabb304-75e8-41d2-b9a4-cd411f5a02d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:35.815626 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cabb304-75e8-41d2-b9a4-cd411f5a02d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:35.815897 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cabb304-75e8-41d2-b9a4-cd411f5a02d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:35.817487 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-6cabb304-75e8-41d2-b9a4-cd411f5a02d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:35.933001 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cabb304-75e8-41d2-b9a4-cd411f5a02d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:57:35.934517 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cabb304-75e8-41d2-b9a4-cd411f5a02d7 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:57:35.956760 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 504/1008] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:57:35 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:57:38.462834 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b020658-5b32-4cc8-bd42-e387b2ddf761 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:38.463434 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b020658-5b32-4cc8-bd42-e387b2ddf761 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:38.463434 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b020658-5b32-4cc8-bd42-e387b2ddf761 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:38.464733 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 505/1009] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:57:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:57:38.469922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee9571-e1c0-4f6b-b624-3b3e48cca5f9 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:38.470385 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee9571-e1c0-4f6b-b624-3b3e48cca5f9 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:38.470519 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee9571-e1c0-4f6b-b624-3b3e48cca5f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:38.533160 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:57:38.536096 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-d4ee9571-e1c0-4f6b-b624-3b3e48cca5f9 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:57:38.537804 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d4ee9571-e1c0-4f6b-b624-3b3e48cca5f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:38.633564 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 505/1010] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:57:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:57:38.912986 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b020658-5b32-4cc8-bd42-e387b2ddf761 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:57:38.914582 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-4b020658-5b32-4cc8-bd42-e387b2ddf761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:39.016829 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b020658-5b32-4cc8-bd42-e387b2ddf761 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:57:39.089493 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28be5b9-a7eb-40a8-a304-5fdffe3a461a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:57:39.090673 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28be5b9-a7eb-40a8-a304-5fdffe3a461a service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:39.090950 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28be5b9-a7eb-40a8-a304-5fdffe3a461a service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:39.091218 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28be5b9-a7eb-40a8-a304-5fdffe3a461a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:39.092789 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-e28be5b9-a7eb-40a8-a304-5fdffe3a461a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:39.118692 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:57:39.198401 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28be5b9-a7eb-40a8-a304-5fdffe3a461a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:57:39.200290 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28be5b9-a7eb-40a8-a304-5fdffe3a461a service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:57:39.218859 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 506/1011] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:57:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:57:41.389649 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6ff81-a76a-456d-8634-77a7d4d764cd None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:41.390209 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6ff81-a76a-456d-8634-77a7d4d764cd None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:41.390209 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6ff81-a76a-456d-8634-77a7d4d764cd None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:41.391718 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 506/1012] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:57:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:57:41.396898 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9aa50-39be-4729-83da-eb7d4963c18f None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:41.397322 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9aa50-39be-4729-83da-eb7d4963c18f None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:41.397520 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d9aa50-39be-4729-83da-eb7d4963c18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:41.462490 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-79d9aa50-39be-4729-83da-eb7d4963c18f None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:57:41.463679 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-79d9aa50-39be-4729-83da-eb7d4963c18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:41.535373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:57:41.557346 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 507/1013] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:57:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 19:57:41.843875 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b6ff81-a76a-456d-8634-77a7d4d764cd None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:57:41.845387 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-02b6ff81-a76a-456d-8634-77a7d4d764cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:41.948241 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b6ff81-a76a-456d-8634-77a7d4d764cd None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:57:41.977714 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:57:42.023308 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3237a0-b33a-40a0-a8de-674eab2589ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:57:42.023915 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3237a0-b33a-40a0-a8de-674eab2589ad service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:42.024195 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3237a0-b33a-40a0-a8de-674eab2589ad service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:42.024466 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3237a0-b33a-40a0-a8de-674eab2589ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:42.026040 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-6b3237a0-b33a-40a0-a8de-674eab2589ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:42.133208 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3237a0-b33a-40a0-a8de-674eab2589ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:57:42.134669 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3237a0-b33a-40a0-a8de-674eab2589ad service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:57:42.152987 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 507/1014] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:57:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:57:44.700012 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ba362-0375-4cf2-8fb4-6031fdc25401 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:44.700574 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ba362-0375-4cf2-8fb4-6031fdc25401 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:44.700574 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345ba362-0375-4cf2-8fb4-6031fdc25401 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:44.701912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 508/1015] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:57:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:57:44.706965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c0b02-4067-4852-a4f8-6dad0177ead9 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:44.707401 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c0b02-4067-4852-a4f8-6dad0177ead9 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:44.707494 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c0b02-4067-4852-a4f8-6dad0177ead9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:44.777270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-bb0c0b02-4067-4852-a4f8-6dad0177ead9 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:57:44.778510 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-bb0c0b02-4067-4852-a4f8-6dad0177ead9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:44.786975 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:57:44.874669 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 508/1016] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:57:44 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:57:45.154793 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ba362-0375-4cf2-8fb4-6031fdc25401 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:57:45.156330 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-345ba362-0375-4cf2-8fb4-6031fdc25401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:45.259060 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345ba362-0375-4cf2-8fb4-6031fdc25401 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:57:45.332977 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f35341-820e-42b6-970b-b460ef9ad86a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:57:45.333585 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f35341-820e-42b6-970b-b460ef9ad86a service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:45.333864 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f35341-820e-42b6-970b-b460ef9ad86a service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:45.334136 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f35341-820e-42b6-970b-b460ef9ad86a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:45.335756 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-b2f35341-820e-42b6-970b-b460ef9ad86a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:45.373135 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:57:45.445911 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f35341-820e-42b6-970b-b460ef9ad86a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:57:45.447372 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f35341-820e-42b6-970b-b460ef9ad86a service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:57:45.465306 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 509/1017] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:57:45 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:57:49.497493 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de132bcd-2e7f-407a-a71e-a1dc533fb827 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:49.498014 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de132bcd-2e7f-407a-a71e-a1dc533fb827 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:49.498014 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de132bcd-2e7f-407a-a71e-a1dc533fb827 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:49.499354 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 509/1018] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:57:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:57:49.504827 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828471f-d5da-4491-a3a2-4d50eb856c4d None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:49.505239 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828471f-d5da-4491-a3a2-4d50eb856c4d None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:49.505378 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a828471f-d5da-4491-a3a2-4d50eb856c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:49.572150 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-a828471f-d5da-4491-a3a2-4d50eb856c4d None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:57:49.573348 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-a828471f-d5da-4491-a3a2-4d50eb856c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:49.656620 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:57:49.668972 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 510/1019] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:57:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:57:49.953083 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de132bcd-2e7f-407a-a71e-a1dc533fb827 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:57:49.954696 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-de132bcd-2e7f-407a-a71e-a1dc533fb827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:50.063533 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de132bcd-2e7f-407a-a71e-a1dc533fb827 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:57:50.100626 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:57:50.138938 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c803d5-afc3-4f86-bddd-f7304e8c7eeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:57:50.139570 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c803d5-afc3-4f86-bddd-f7304e8c7eeb service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:50.139846 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c803d5-afc3-4f86-bddd-f7304e8c7eeb service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:50.140112 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c803d5-afc3-4f86-bddd-f7304e8c7eeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:50.141676 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d5c803d5-afc3-4f86-bddd-f7304e8c7eeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:50.252024 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c803d5-afc3-4f86-bddd-f7304e8c7eeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:57:50.253372 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c803d5-afc3-4f86-bddd-f7304e8c7eeb service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:57:50.272322 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 510/1020] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:57:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:57:52.987154 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf8614-f1eb-4eb3-93cc-1901e1ff8a75 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:52.987711 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf8614-f1eb-4eb3-93cc-1901e1ff8a75 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:52.987711 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faaf8614-f1eb-4eb3-93cc-1901e1ff8a75 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:52.989136 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 511/1021] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:57:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:57:52.994453 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8eed9-eb4c-43b6-9613-c10aedec6024 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:52.994828 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8eed9-eb4c-43b6-9613-c10aedec6024 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:52.995024 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b8eed9-eb4c-43b6-9613-c10aedec6024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:53.063519 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-06b8eed9-eb4c-43b6-9613-c10aedec6024 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:57:53.064721 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-06b8eed9-eb4c-43b6-9613-c10aedec6024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:53.082633 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:57:53.163898 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 511/1022] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:57:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:57:53.447783 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf8614-f1eb-4eb3-93cc-1901e1ff8a75 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:57:53.449305 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-faaf8614-f1eb-4eb3-93cc-1901e1ff8a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:53.557246 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faaf8614-f1eb-4eb3-93cc-1901e1ff8a75 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:57:53.632631 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43916c96-0a91-4631-990a-f583dbb902f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:57:53.633234 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43916c96-0a91-4631-990a-f583dbb902f8 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:53.633512 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43916c96-0a91-4631-990a-f583dbb902f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:53.633781 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43916c96-0a91-4631-990a-f583dbb902f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:53.635409 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-43916c96-0a91-4631-990a-f583dbb902f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:53.679750 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:57:53.746404 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43916c96-0a91-4631-990a-f583dbb902f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:57:53.747845 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43916c96-0a91-4631-990a-f583dbb902f8 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:57:53.771551 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 512/1023] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:57:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:57:57.916924 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3df99-8831-4e2d-a33d-d1bc404eeebd None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:57.917420 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3df99-8831-4e2d-a33d-d1bc404eeebd None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:57.917507 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc3df99-8831-4e2d-a33d-d1bc404eeebd None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:57.919387 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 512/1024] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:57:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:57:57.924519 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54a6ec-0598-4894-8a04-80e1bcbb4e2e None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:57.924918 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54a6ec-0598-4894-8a04-80e1bcbb4e2e None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:57.925062 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b54a6ec-0598-4894-8a04-80e1bcbb4e2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:57.994816 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-8b54a6ec-0598-4894-8a04-80e1bcbb4e2e None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:57:57.996869 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-8b54a6ec-0598-4894-8a04-80e1bcbb4e2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:58.095769 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:57:58.098111 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 513/1025] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:57:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:57:58.385216 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc3df99-8831-4e2d-a33d-d1bc404eeebd None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:57:58.387344 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-8dc3df99-8831-4e2d-a33d-d1bc404eeebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:58.488509 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc3df99-8831-4e2d-a33d-d1bc404eeebd None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:57:58.536637 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:57:58.563220 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d8a244-2a95-483f-b312-1f734ceb1b08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:57:58.563827 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d8a244-2a95-483f-b312-1f734ceb1b08 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:57:58.564107 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d8a244-2a95-483f-b312-1f734ceb1b08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:57:58.564383 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d8a244-2a95-483f-b312-1f734ceb1b08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:57:58.565968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-86d8a244-2a95-483f-b312-1f734ceb1b08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:57:58.673386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d8a244-2a95-483f-b312-1f734ceb1b08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:57:58.674776 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d8a244-2a95-483f-b312-1f734ceb1b08 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:57:58.692711 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 513/1026] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:57:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:58:03.535192 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048e934-ef17-498c-80ca-eb9054e67b9e None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:03.535719 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048e934-ef17-498c-80ca-eb9054e67b9e None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:03.535719 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e048e934-ef17-498c-80ca-eb9054e67b9e None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:03.543618 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a4be2-4788-496d-a496-d5f7983c9fae None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:03.544888 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a4be2-4788-496d-a496-d5f7983c9fae None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:03.545513 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8a4be2-4788-496d-a496-d5f7983c9fae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:03.546280 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 514/1027] 10.222.0.13 () {54 vars in 921 bytes} [Sun Oct 23 19:58:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:58:03.655363 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-bb8a4be2-4788-496d-a496-d5f7983c9fae None None] MFA Rules not processed for user `312489c49a304ca9aacebd9843a4b65f`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:58:03.656628 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-bb8a4be2-4788-496d-a496-d5f7983c9fae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:03.690616 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:03.812697 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 514/1028] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Oct 23 19:58:03 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:58:07.906301 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31fec8d-5a50-4908-9f8c-82a4d0898cd8 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:07.907350 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31fec8d-5a50-4908-9f8c-82a4d0898cd8 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:07.907781 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31fec8d-5a50-4908-9f8c-82a4d0898cd8 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:07.909325 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 515/1029] 10.222.0.13 () {54 vars in 921 bytes} [Sun Oct 23 19:58:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:58:07.915362 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36230f-bf53-49b2-922c-56b6586631d1 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:07.915836 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36230f-bf53-49b2-922c-56b6586631d1 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:07.915931 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36230f-bf53-49b2-922c-56b6586631d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:07.996680 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-ff36230f-bf53-49b2-922c-56b6586631d1 None None] MFA Rules not processed for user `312489c49a304ca9aacebd9843a4b65f`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:58:07.997983 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ff36230f-bf53-49b2-922c-56b6586631d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:08.077261 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:08.121966 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 515/1030] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Oct 23 19:58:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:58:09.517749 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a87df1-bf05-4772-9325-b07b626e74a2 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:09.518324 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a87df1-bf05-4772-9325-b07b626e74a2 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:09.518324 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a87df1-bf05-4772-9325-b07b626e74a2 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:09.519720 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 516/1031] 10.222.0.13 () {54 vars in 921 bytes} [Sun Oct 23 19:58:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:58:09.525094 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611070e-78b7-425a-9b1d-9914182fcdc0 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:09.525518 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611070e-78b7-425a-9b1d-9914182fcdc0 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:09.525649 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0611070e-78b7-425a-9b1d-9914182fcdc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:09.603186 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-0611070e-78b7-425a-9b1d-9914182fcdc0 None None] MFA Rules not processed for user `312489c49a304ca9aacebd9843a4b65f`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:58:09.604392 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-0611070e-78b7-425a-9b1d-9914182fcdc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:09.714016 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 516/1032] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Oct 23 19:58:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:58:13.207493 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92fcaf-c767-4fd5-9554-5d2797ab8e9d None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:13.208576 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92fcaf-c767-4fd5-9554-5d2797ab8e9d None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:13.209817 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca92fcaf-c767-4fd5-9554-5d2797ab8e9d None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:13.211212 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 517/1033] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:58:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:58:13.217294 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c30243-c410-4383-8db3-c3acb8161407 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:13.217718 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c30243-c410-4383-8db3-c3acb8161407 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:13.218111 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c30243-c410-4383-8db3-c3acb8161407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:13.255222 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:13.289514 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-c1c30243-c410-4383-8db3-c3acb8161407 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:58:13.290862 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c1c30243-c410-4383-8db3-c3acb8161407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:13.393314 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 517/1034] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:58:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:58:13.694102 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931ac19-446a-429e-a479-abcb925cedbd None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:13.694797 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931ac19-446a-429e-a479-abcb925cedbd None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:13.695301 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931ac19-446a-429e-a479-abcb925cedbd None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:13.697572 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 518/1035] 10.222.0.13 () {54 vars in 961 bytes} [Sun Oct 23 19:58:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:58:13.704798 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff675d2-643d-458e-bc43-f77f185d5571 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:13.705210 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff675d2-643d-458e-bc43-f77f185d5571 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:13.705396 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff675d2-643d-458e-bc43-f77f185d5571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:13.792172 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-6ff675d2-643d-458e-bc43-f77f185d5571 None None] MFA Rules not processed for user `0b3d512304254ffcaa8dad90abf1f9c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:58:13.793947 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-6ff675d2-643d-458e-bc43-f77f185d5571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:13.848732 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:13.922067 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 518/1036] 10.222.0.13 () {58 vars in 1058 bytes} [Sun Oct 23 19:58:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:58:13.929747 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931ac19-446a-429e-a479-abcb925cedbd None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:58:13.931355 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-c931ac19-446a-429e-a479-abcb925cedbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:14.087881 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c931ac19-446a-429e-a479-abcb925cedbd None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:58:14.175198 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:14.221058 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172425a-ba35-44fc-b381-248e39f7b040 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:58:14.222326 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172425a-ba35-44fc-b381-248e39f7b040 service nova] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:14.222707 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172425a-ba35-44fc-b381-248e39f7b040 service nova] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:14.223084 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172425a-ba35-44fc-b381-248e39f7b040 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:14.224767 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-9172425a-ba35-44fc-b381-248e39f7b040 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:14.327984 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9172425a-ba35-44fc-b381-248e39f7b040 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:58:14.329729 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9172425a-ba35-44fc-b381-248e39f7b040 service nova] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:58:14.349613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 519/1037] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:58:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:58:16.705373 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e1b73-7c52-4020-a1e4-7b803dbfcfd7 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:16.705824 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e1b73-7c52-4020-a1e4-7b803dbfcfd7 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:16.705824 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1e1b73-7c52-4020-a1e4-7b803dbfcfd7 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:16.711273 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 519/1038] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:58:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:58:16.712457 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e1cd7-898b-41a6-9e6a-53882ecb3d08 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:16.713161 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e1cd7-898b-41a6-9e6a-53882ecb3d08 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:16.713161 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735e1cd7-898b-41a6-9e6a-53882ecb3d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:16.797113 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-735e1cd7-898b-41a6-9e6a-53882ecb3d08 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:58:16.798376 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-735e1cd7-898b-41a6-9e6a-53882ecb3d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:16.841562 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:16.904525 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 520/1039] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:58:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 19:58:17.195647 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e1b73-7c52-4020-a1e4-7b803dbfcfd7 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:58:17.197265 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ac1e1b73-7c52-4020-a1e4-7b803dbfcfd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:17.376550 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1e1b73-7c52-4020-a1e4-7b803dbfcfd7 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:58:17.389756 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:17.515343 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed27164c-4cf9-4ecc-9158-e03f538844f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:58:17.515952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27164c-4cf9-4ecc-9158-e03f538844f3 service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:17.516228 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27164c-4cf9-4ecc-9158-e03f538844f3 service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:17.516498 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27164c-4cf9-4ecc-9158-e03f538844f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:17.518079 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ed27164c-4cf9-4ecc-9158-e03f538844f3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:17.609427 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed27164c-4cf9-4ecc-9158-e03f538844f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:58:17.611080 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed27164c-4cf9-4ecc-9158-e03f538844f3 service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:58:17.629132 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 520/1040] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:58:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:58:20.113243 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d889a5-f79b-4727-91bd-56ee5271815a None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:20.113771 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d889a5-f79b-4727-91bd-56ee5271815a None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:20.113771 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d889a5-f79b-4727-91bd-56ee5271815a None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:20.115449 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 521/1041] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:58:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:58:20.121200 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931a7cd-603a-47c0-a37f-fa396fc36edf None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:20.121621 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931a7cd-603a-47c0-a37f-fa396fc36edf None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:20.121921 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931a7cd-603a-47c0-a37f-fa396fc36edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:20.139488 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:20.196666 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-0931a7cd-603a-47c0-a37f-fa396fc36edf None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:58:20.197930 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-0931a7cd-603a-47c0-a37f-fa396fc36edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:20.301187 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 521/1042] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:58:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:58:20.588629 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6886d-f3d2-4654-8691-56085ae1a4b0 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:20.589041 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6886d-f3d2-4654-8691-56085ae1a4b0 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:20.589569 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6886d-f3d2-4654-8691-56085ae1a4b0 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:20.591209 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 522/1043] 10.222.0.13 () {54 vars in 961 bytes} [Sun Oct 23 19:58:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:58:20.596819 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb39f1-28d1-46fb-a62f-cfc185af17a1 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:20.597104 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb39f1-28d1-46fb-a62f-cfc185af17a1 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:20.597379 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb39f1-28d1-46fb-a62f-cfc185af17a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:20.676236 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-5efb39f1-28d1-46fb-a62f-cfc185af17a1 None None] MFA Rules not processed for user `0b3d512304254ffcaa8dad90abf1f9c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:58:20.677420 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5efb39f1-28d1-46fb-a62f-cfc185af17a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:20.713321 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:20.801146 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 522/1044] 10.222.0.13 () {58 vars in 1058 bytes} [Sun Oct 23 19:58:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:58:20.809684 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6886d-f3d2-4654-8691-56085ae1a4b0 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:58:20.811315 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-fcd6886d-f3d2-4654-8691-56085ae1a4b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:20.962038 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6886d-f3d2-4654-8691-56085ae1a4b0 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:58:20.982859 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:21.091830 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127985a-f1a1-48bb-91a5-699c4f427349 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:58:21.092734 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127985a-f1a1-48bb-91a5-699c4f427349 service nova] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:21.093168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127985a-f1a1-48bb-91a5-699c4f427349 service nova] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:21.093587 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127985a-f1a1-48bb-91a5-699c4f427349 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:21.095513 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-4127985a-f1a1-48bb-91a5-699c4f427349 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:21.197056 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4127985a-f1a1-48bb-91a5-699c4f427349 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:58:21.198818 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4127985a-f1a1-48bb-91a5-699c4f427349 service nova] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:58:21.217700 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 523/1045] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:58:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 19:58:21.367874 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0a627-3e3b-4fca-9da4-bee47a96d5e2 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:21.368436 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0a627-3e3b-4fca-9da4-bee47a96d5e2 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:21.368436 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f0a627-3e3b-4fca-9da4-bee47a96d5e2 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:21.370019 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 523/1046] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:58:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:58:21.375784 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395cacb2-dae9-4ff9-839b-9582c95867b2 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:21.376069 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395cacb2-dae9-4ff9-839b-9582c95867b2 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:21.376362 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395cacb2-dae9-4ff9-839b-9582c95867b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:21.423891 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:21.462855 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-395cacb2-dae9-4ff9-839b-9582c95867b2 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:58:21.464158 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-395cacb2-dae9-4ff9-839b-9582c95867b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:21.572399 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 524/1047] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:58:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 19:58:21.906823 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf1a45-f54c-48d0-93bc-0fae3a1ab276 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:21.907108 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf1a45-f54c-48d0-93bc-0fae3a1ab276 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:21.907383 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bf1a45-f54c-48d0-93bc-0fae3a1ab276 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:21.910285 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 524/1048] 10.222.0.13 () {54 vars in 963 bytes} [Sun Oct 23 19:58:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:58:21.914114 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485b8f0-2687-4c52-a8c2-ce25464d5012 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:21.914504 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485b8f0-2687-4c52-a8c2-ce25464d5012 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:21.914781 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485b8f0-2687-4c52-a8c2-ce25464d5012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:21.999776 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-9485b8f0-2687-4c52-a8c2-ce25464d5012 None None] MFA Rules not processed for user `7bcc5f5101fb4913aaf5caf564ca6b60`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:58:22.001123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-9485b8f0-2687-4c52-a8c2-ce25464d5012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:22.054239 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:22.086929 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 525/1049] 10.222.0.13 () {58 vars in 1060 bytes} [Sun Oct 23 19:58:21 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:58:22.095677 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf1a45-f54c-48d0-93bc-0fae3a1ab276 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:58:22.097316 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-05bf1a45-f54c-48d0-93bc-0fae3a1ab276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:22.217092 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bf1a45-f54c-48d0-93bc-0fae3a1ab276 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:58:22.264030 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:22.305056 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af03359-aab8-41ed-981f-2ef641aaac84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:58:22.305568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af03359-aab8-41ed-981f-2ef641aaac84 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:22.305892 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af03359-aab8-41ed-981f-2ef641aaac84 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:22.306193 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af03359-aab8-41ed-981f-2ef641aaac84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:22.307722 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3af03359-aab8-41ed-981f-2ef641aaac84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:22.427795 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af03359-aab8-41ed-981f-2ef641aaac84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:58:22.429157 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af03359-aab8-41ed-981f-2ef641aaac84 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:58:22.450676 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 525/1050] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:58:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:58:23.456568 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd146b-495d-43ee-80e3-37dcfa90d51c None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:23.457105 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd146b-495d-43ee-80e3-37dcfa90d51c None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:23.457105 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd146b-495d-43ee-80e3-37dcfa90d51c None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:23.458517 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 526/1051] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:58:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:58:23.463687 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1e05c-44c6-4a6a-a026-4036bd497b19 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:23.464092 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1e05c-44c6-4a6a-a026-4036bd497b19 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:23.464340 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1e05c-44c6-4a6a-a026-4036bd497b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:23.529903 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-d2a1e05c-44c6-4a6a-a026-4036bd497b19 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:58:23.531133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d2a1e05c-44c6-4a6a-a026-4036bd497b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:23.551203 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:23.628117 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 526/1052] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:58:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:58:23.919747 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd146b-495d-43ee-80e3-37dcfa90d51c None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:58:23.921340 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-22fd146b-495d-43ee-80e3-37dcfa90d51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:24.088631 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd146b-495d-43ee-80e3-37dcfa90d51c None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:58:24.143463 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:24.212727 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d39a20-8dc7-43e1-8b14-31a7fc16fa4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:58:24.213339 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d39a20-8dc7-43e1-8b14-31a7fc16fa4a service nova] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:24.213623 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d39a20-8dc7-43e1-8b14-31a7fc16fa4a service nova] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:24.213891 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d39a20-8dc7-43e1-8b14-31a7fc16fa4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:24.215594 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-19d39a20-8dc7-43e1-8b14-31a7fc16fa4a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:24.310573 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d39a20-8dc7-43e1-8b14-31a7fc16fa4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:58:24.312176 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d39a20-8dc7-43e1-8b14-31a7fc16fa4a service nova] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:58:24.331109 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 527/1053] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:58:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:58:26.544970 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917b29f-25a5-42e7-8fb4-170089d778b1 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:26.545553 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917b29f-25a5-42e7-8fb4-170089d778b1 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:26.545553 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e917b29f-25a5-42e7-8fb4-170089d778b1 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:26.547100 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 527/1054] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:58:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:58:26.552144 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c5e65-14db-42f6-929b-941e300e5cda None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:26.552582 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c5e65-14db-42f6-929b-941e300e5cda None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:26.552685 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7c5e65-14db-42f6-929b-941e300e5cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:26.628181 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-7d7c5e65-14db-42f6-929b-941e300e5cda None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:58:26.629298 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-7d7c5e65-14db-42f6-929b-941e300e5cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:26.636251 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:26.737400 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 528/1055] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:58:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:58:27.028958 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e917b29f-25a5-42e7-8fb4-170089d778b1 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:58:27.030498 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e917b29f-25a5-42e7-8fb4-170089d778b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:27.185077 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e917b29f-25a5-42e7-8fb4-170089d778b1 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:58:27.189363 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:27.303282 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e23265d-cd81-481d-890d-e8d2272b0704 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:58:27.303796 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e23265d-cd81-481d-890d-e8d2272b0704 service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:27.304078 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e23265d-cd81-481d-890d-e8d2272b0704 service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:27.304354 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e23265d-cd81-481d-890d-e8d2272b0704 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:27.305932 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-0e23265d-cd81-481d-890d-e8d2272b0704 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:27.403312 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e23265d-cd81-481d-890d-e8d2272b0704 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:58:27.404911 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e23265d-cd81-481d-890d-e8d2272b0704 service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:58:27.423584 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 528/1056] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:58:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:58:29.632284 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df11563a-f266-44a9-b16b-3eba75ebafd4 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:29.632847 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df11563a-f266-44a9-b16b-3eba75ebafd4 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:29.633136 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df11563a-f266-44a9-b16b-3eba75ebafd4 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:29.634778 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 529/1057] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:58:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:58:29.640354 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fe6f7-084f-4c3d-a685-48e73d9e6d36 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:29.640805 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fe6f7-084f-4c3d-a685-48e73d9e6d36 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:29.640903 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9fe6f7-084f-4c3d-a685-48e73d9e6d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:29.652753 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:29.711130 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-5b9fe6f7-084f-4c3d-a685-48e73d9e6d36 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:58:29.712317 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5b9fe6f7-084f-4c3d-a685-48e73d9e6d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:29.855838 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 529/1058] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:58:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:58:30.146771 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df11563a-f266-44a9-b16b-3eba75ebafd4 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:58:30.148287 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-df11563a-f266-44a9-b16b-3eba75ebafd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:30.330036 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:30.344367 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df11563a-f266-44a9-b16b-3eba75ebafd4 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:58:30.485187 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f852b077-1b49-4adb-8755-fb5adf3a1dce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:58:30.485838 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852b077-1b49-4adb-8755-fb5adf3a1dce service nova] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:30.486138 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852b077-1b49-4adb-8755-fb5adf3a1dce service nova] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:30.486456 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f852b077-1b49-4adb-8755-fb5adf3a1dce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:30.488201 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-f852b077-1b49-4adb-8755-fb5adf3a1dce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:30.585140 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f852b077-1b49-4adb-8755-fb5adf3a1dce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:58:30.586751 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f852b077-1b49-4adb-8755-fb5adf3a1dce service nova] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:58:30.598543 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:30.609992 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 530/1059] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:58:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 19:58:33.215296 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28467422-4e36-419a-8988-6cf26ed0160f None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:33.215830 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28467422-4e36-419a-8988-6cf26ed0160f None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:33.216081 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28467422-4e36-419a-8988-6cf26ed0160f None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:33.218162 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 530/1060] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:58:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:58:33.225398 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1cf0a6-ce13-4f9b-8f9f-7e2a11422243 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:33.225865 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1cf0a6-ce13-4f9b-8f9f-7e2a11422243 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:33.226221 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1cf0a6-ce13-4f9b-8f9f-7e2a11422243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:33.312149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-5b1cf0a6-ce13-4f9b-8f9f-7e2a11422243 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:58:33.313286 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-5b1cf0a6-ce13-4f9b-8f9f-7e2a11422243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:33.430151 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 531/1061] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:58:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:58:33.716755 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28467422-4e36-419a-8988-6cf26ed0160f None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:58:33.718279 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-28467422-4e36-419a-8988-6cf26ed0160f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:33.805044 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:33.886052 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28467422-4e36-419a-8988-6cf26ed0160f None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:58:34.024858 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43139820-d8c9-4df5-93cf-5511da2585a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:58:34.025622 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43139820-d8c9-4df5-93cf-5511da2585a6 service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:34.025975 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43139820-d8c9-4df5-93cf-5511da2585a6 service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:34.026360 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43139820-d8c9-4df5-93cf-5511da2585a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:34.029042 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-43139820-d8c9-4df5-93cf-5511da2585a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:34.093752 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:34.143136 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43139820-d8c9-4df5-93cf-5511da2585a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:58:34.144716 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43139820-d8c9-4df5-93cf-5511da2585a6 service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:58:34.163536 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 531/1062] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:58:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:58:36.340680 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da432646-54a9-4783-aa96-996b2fc04353 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:36.341232 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da432646-54a9-4783-aa96-996b2fc04353 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:36.341232 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da432646-54a9-4783-aa96-996b2fc04353 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:36.342658 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 532/1063] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:58:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:58:36.348642 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2f74f-e594-4080-81fb-47cc191e3d37 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:36.349129 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2f74f-e594-4080-81fb-47cc191e3d37 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:36.349221 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec2f74f-e594-4080-81fb-47cc191e3d37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:36.415361 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-2ec2f74f-e594-4080-81fb-47cc191e3d37 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:58:36.416639 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2ec2f74f-e594-4080-81fb-47cc191e3d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:36.514492 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 532/1064] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:58:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:58:36.799276 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da432646-54a9-4783-aa96-996b2fc04353 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:58:36.800849 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-da432646-54a9-4783-aa96-996b2fc04353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:36.879985 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:36.950389 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da432646-54a9-4783-aa96-996b2fc04353 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:58:37.067575 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b332848-8fc8-459f-a3b9-80e3e48d9c69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:58:37.068185 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b332848-8fc8-459f-a3b9-80e3e48d9c69 service nova] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:37.068463 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b332848-8fc8-459f-a3b9-80e3e48d9c69 service nova] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:37.068734 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b332848-8fc8-459f-a3b9-80e3e48d9c69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:37.070607 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-5b332848-8fc8-459f-a3b9-80e3e48d9c69 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:37.109736 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:37.162874 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b332848-8fc8-459f-a3b9-80e3e48d9c69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:58:37.164468 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b332848-8fc8-459f-a3b9-80e3e48d9c69 service nova] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:58:37.183071 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 533/1065] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:58:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:58:37.596726 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac7506-d85f-49fb-b2db-0d3461b6aa0e None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:37.597265 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac7506-d85f-49fb-b2db-0d3461b6aa0e None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:37.597265 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac7506-d85f-49fb-b2db-0d3461b6aa0e None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:37.598680 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 533/1066] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:58:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:58:37.604067 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29b20f-c859-4bfb-93f7-6fe9bd99b2a6 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:37.604467 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29b20f-c859-4bfb-93f7-6fe9bd99b2a6 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:37.604641 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a29b20f-c859-4bfb-93f7-6fe9bd99b2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:37.670456 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-7a29b20f-c859-4bfb-93f7-6fe9bd99b2a6 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:58:37.671665 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-7a29b20f-c859-4bfb-93f7-6fe9bd99b2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:37.768336 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 534/1067] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:58:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:58:38.073425 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf58e40-75c6-406b-a649-d57edd928337 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:38.073693 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf58e40-75c6-406b-a649-d57edd928337 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:38.073998 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf58e40-75c6-406b-a649-d57edd928337 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:38.075379 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 534/1068] 10.222.0.13 () {54 vars in 962 bytes} [Sun Oct 23 19:58:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:58:38.080732 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b4536-d09d-4f56-9f32-878f86c2c7ef None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:38.081026 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b4536-d09d-4f56-9f32-878f86c2c7ef None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:38.081328 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407b4536-d09d-4f56-9f32-878f86c2c7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:38.098792 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:38.161622 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-407b4536-d09d-4f56-9f32-878f86c2c7ef None None] MFA Rules not processed for user `021216f90d7f4ff6b6eb7a1d86399d99`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:58:38.162845 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-407b4536-d09d-4f56-9f32-878f86c2c7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:38.236817 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 535/1069] 10.222.0.13 () {58 vars in 1059 bytes} [Sun Oct 23 19:58:38 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:58:38.244512 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf58e40-75c6-406b-a649-d57edd928337 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:58:38.246595 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-baf58e40-75c6-406b-a649-d57edd928337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:38.279398 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:38.378186 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf58e40-75c6-406b-a649-d57edd928337 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:58:38.452872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401dd9c0-7129-4f49-880f-ff905de61461 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '021216f90d7f4ff6b6eb7a1d86399d99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:58:38.453479 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dd9c0-7129-4f49-880f-ff905de61461 service glance] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:38.453759 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dd9c0-7129-4f49-880f-ff905de61461 service glance] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:38.454035 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401dd9c0-7129-4f49-880f-ff905de61461 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:38.455647 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-401dd9c0-7129-4f49-880f-ff905de61461 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:38.534983 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:38.567875 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401dd9c0-7129-4f49-880f-ff905de61461 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:58:38.569572 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401dd9c0-7129-4f49-880f-ff905de61461 service glance] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:58:38.587141 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 535/1070] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:58:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:58:39.330609 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ab07b-fb64-4550-b3a3-bf0ec4f5f245 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:39.331089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ab07b-fb64-4550-b3a3-bf0ec4f5f245 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:39.331190 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152ab07b-fb64-4550-b3a3-bf0ec4f5f245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:39.399203 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-152ab07b-fb64-4550-b3a3-bf0ec4f5f245 None None] MFA Rules not processed for user `ff5cfe4d31614e2db9059c69116297cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:58:39.400213 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57165c3b-9d3e-4b9e-b994-221016cec69e None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:39.400655 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-152ab07b-fb64-4550-b3a3-bf0ec4f5f245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:39.400780 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57165c3b-9d3e-4b9e-b994-221016cec69e None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:39.400780 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57165c3b-9d3e-4b9e-b994-221016cec69e None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:39.402660 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 536/1071] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 19:58:39.407545 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac43f6-1927-4b2f-b215-b07800b0ef42 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:39.407828 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac43f6-1927-4b2f-b215-b07800b0ef42 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:39.408109 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac43f6-1927-4b2f-b215-b07800b0ef42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:39.459425 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:39.475074 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-a8ac43f6-1927-4b2f-b215-b07800b0ef42 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:58:39.476275 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a8ac43f6-1927-4b2f-b215-b07800b0ef42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:39.491281 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 536/1072] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Oct 23 19:58:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:58:39.537889 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83fe89e-e960-43bf-81db-fd131f1419bf None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:39.538184 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83fe89e-e960-43bf-81db-fd131f1419bf None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:39.538498 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83fe89e-e960-43bf-81db-fd131f1419bf None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:39.539844 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 537/1073] 10.222.0.13 () {54 vars in 943 bytes} [Sun Oct 23 19:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 19:58:39.546391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e640ca-5995-4fc0-9dbb-13f3a8c7e312 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:39.546672 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e640ca-5995-4fc0-9dbb-13f3a8c7e312 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:39.546945 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e640ca-5995-4fc0-9dbb-13f3a8c7e312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:39.579008 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 537/1074] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:58:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 19:58:39.623031 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-e3e640ca-5995-4fc0-9dbb-13f3a8c7e312 None None] MFA Rules not processed for user `15f603d1e7f54751aa79317e2ea00a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:58:39.624235 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-e3e640ca-5995-4fc0-9dbb-13f3a8c7e312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:39.731373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 538/1075] 10.222.0.13 () {58 vars in 1040 bytes} [Sun Oct 23 19:58:39 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 19:58:39.740583 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac43f6-1927-4b2f-b215-b07800b0ef42 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:58:39.741831 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a8ac43f6-1927-4b2f-b215-b07800b0ef42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:39.782577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:39.845986 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac43f6-1927-4b2f-b215-b07800b0ef42 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:58:39.870179 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e640ca-5995-4fc0-9dbb-13f3a8c7e312 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:58:39.871460 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-e3e640ca-5995-4fc0-9dbb-13f3a8c7e312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:39.902526 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:39.951325 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad3a4e-9bfe-4dff-acb7-b4c936396612 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15f603d1e7f54751aa79317e2ea00a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:58:39.951929 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad3a4e-9bfe-4dff-acb7-b4c936396612 service swift] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:39.952203 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad3a4e-9bfe-4dff-acb7-b4c936396612 service swift] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:39.952473 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad3a4e-9bfe-4dff-acb7-b4c936396612 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:39.954091 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-78ad3a4e-9bfe-4dff-acb7-b4c936396612 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:39.971127 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e640ca-5995-4fc0-9dbb-13f3a8c7e312 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:58:40.008774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:40.041477 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ad3a4e-9bfe-4dff-acb7-b4c936396612 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:58:40.043623 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ad3a4e-9bfe-4dff-acb7-b4c936396612 service swift] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:58:40.051912 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 538/1076] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Oct 23 19:58:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 19:58:40.098466 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279c2727-5ed9-4afe-b154-87d7586d664f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:58:40.099084 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c2727-5ed9-4afe-b154-87d7586d664f service nova] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:40.099362 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c2727-5ed9-4afe-b154-87d7586d664f service nova] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:40.099630 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279c2727-5ed9-4afe-b154-87d7586d664f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:40.101261 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-279c2727-5ed9-4afe-b154-87d7586d664f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:40.148111 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:40.209418 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279c2727-5ed9-4afe-b154-87d7586d664f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:58:40.211051 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279c2727-5ed9-4afe-b154-87d7586d664f service nova] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:58:40.231208 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 539/1077] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:58:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 19:58:42.457306 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050f296-f647-4b57-8296-d0b61c267348 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:42.457911 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050f296-f647-4b57-8296-d0b61c267348 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:42.457911 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a050f296-f647-4b57-8296-d0b61c267348 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:42.459251 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 539/1078] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:58:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:58:42.465010 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c72948d-b4df-452e-80a2-5c636ead7d74 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:42.465451 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c72948d-b4df-452e-80a2-5c636ead7d74 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:42.465558 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c72948d-b4df-452e-80a2-5c636ead7d74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:42.533578 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-4c72948d-b4df-452e-80a2-5c636ead7d74 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:58:42.534835 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-4c72948d-b4df-452e-80a2-5c636ead7d74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:42.641566 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 540/1079] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:58:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 19:58:42.929702 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a050f296-f647-4b57-8296-d0b61c267348 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:58:42.931325 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a050f296-f647-4b57-8296-d0b61c267348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:43.100469 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a050f296-f647-4b57-8296-d0b61c267348 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:58:43.157111 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:43.226861 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ec4d70-5825-4777-9c6c-df322e901b06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:58:43.227748 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec4d70-5825-4777-9c6c-df322e901b06 service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:43.228186 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec4d70-5825-4777-9c6c-df322e901b06 service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:43.228626 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ec4d70-5825-4777-9c6c-df322e901b06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:43.231583 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-21ec4d70-5825-4777-9c6c-df322e901b06 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:43.329377 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ec4d70-5825-4777-9c6c-df322e901b06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:58:43.331007 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ec4d70-5825-4777-9c6c-df322e901b06 service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:58:43.350394 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 540/1080] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:58:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:58:45.660869 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a05194-0128-4b26-8526-cff1898654aa None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:45.661399 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a05194-0128-4b26-8526-cff1898654aa None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:45.661399 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a05194-0128-4b26-8526-cff1898654aa None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:45.666668 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 541/1081] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:58:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:58:45.668454 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692fe75-9594-40ab-88e7-57bb9185091b None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:45.668935 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692fe75-9594-40ab-88e7-57bb9185091b None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:45.669454 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2692fe75-9594-40ab-88e7-57bb9185091b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:45.747165 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-2692fe75-9594-40ab-88e7-57bb9185091b None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:58:45.748452 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2692fe75-9594-40ab-88e7-57bb9185091b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:45.751412 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:45.870513 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 541/1082] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:58:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 19:58:46.154894 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a05194-0128-4b26-8526-cff1898654aa None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:58:46.156424 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-b7a05194-0128-4b26-8526-cff1898654aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:46.177161 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:46.317656 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a05194-0128-4b26-8526-cff1898654aa None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:58:46.432632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:46.462259 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8264c22-1df3-4250-9136-245aa6b1d9cc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:58:46.462895 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8264c22-1df3-4250-9136-245aa6b1d9cc service nova] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:46.463175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8264c22-1df3-4250-9136-245aa6b1d9cc service nova] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:46.463445 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8264c22-1df3-4250-9136-245aa6b1d9cc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:46.465083 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-d8264c22-1df3-4250-9136-245aa6b1d9cc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:46.563772 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8264c22-1df3-4250-9136-245aa6b1d9cc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:58:46.565484 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8264c22-1df3-4250-9136-245aa6b1d9cc service nova] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:58:46.588565 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 542/1083] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:58:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:58:48.829659 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16ccc0-e1d7-4463-a915-19d34932c572 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:48.830210 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16ccc0-e1d7-4463-a915-19d34932c572 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:48.830210 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f16ccc0-e1d7-4463-a915-19d34932c572 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:48.835509 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 542/1084] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 19:58:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:58:48.837072 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ecd09-5aa8-45e3-ac41-c26564a08c7d None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:48.837509 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ecd09-5aa8-45e3-ac41-c26564a08c7d None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:48.837682 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ecd09-5aa8-45e3-ac41-c26564a08c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:48.910988 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-493ecd09-5aa8-45e3-ac41-c26564a08c7d None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:58:48.912229 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-493ecd09-5aa8-45e3-ac41-c26564a08c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:48.974534 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:49.028872 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 543/1085] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 19:58:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:58:49.325275 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16ccc0-e1d7-4463-a915-19d34932c572 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:58:49.327110 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3f16ccc0-e1d7-4463-a915-19d34932c572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:49.488002 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:49.504231 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f16ccc0-e1d7-4463-a915-19d34932c572 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:58:49.628762 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7424d390-3fc8-4794-b682-e2f8eccd058f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:58:49.629369 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424d390-3fc8-4794-b682-e2f8eccd058f service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:58:49.629653 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424d390-3fc8-4794-b682-e2f8eccd058f service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:58:49.629931 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7424d390-3fc8-4794-b682-e2f8eccd058f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:58:49.631578 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7424d390-3fc8-4794-b682-e2f8eccd058f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:58:49.733479 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7424d390-3fc8-4794-b682-e2f8eccd058f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:58:49.735106 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7424d390-3fc8-4794-b682-e2f8eccd058f service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:58:49.744859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:49.754946 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 543/1086] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:58:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:58:59.027044 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=19 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:58:59.752401 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:02.828706 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe48fb-26fc-4bfd-95fd-d32ec37af80c None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:02.829230 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe48fb-26fc-4bfd-95fd-d32ec37af80c None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:02.829230 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefe48fb-26fc-4bfd-95fd-d32ec37af80c None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:02.830638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 544/1087] 10.222.0.13 () {54 vars in 918 bytes} [Sun Oct 23 19:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:59:02.835875 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee85e01-b77f-4d5d-afbd-d922946220f9 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:02.836305 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee85e01-b77f-4d5d-afbd-d922946220f9 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:02.836479 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee85e01-b77f-4d5d-afbd-d922946220f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:02.912341 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-6ee85e01-b77f-4d5d-afbd-d922946220f9 None None] MFA Rules not processed for user `312489c49a304ca9aacebd9843a4b65f`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:59:02.913527 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-6ee85e01-b77f-4d5d-afbd-d922946220f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:03.019476 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 544/1088] 10.222.0.13 () {58 vars in 1015 bytes} [Sun Oct 23 19:59:02 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:59:03.033326 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0d7bc-9236-422f-8c25-316bb7c00590 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:03.033607 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0d7bc-9236-422f-8c25-316bb7c00590 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:03.033905 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c0d7bc-9236-422f-8c25-316bb7c00590 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:03.035277 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 545/1089] 10.222.0.13 () {54 vars in 918 bytes} [Sun Oct 23 19:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:59:03.039645 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07ec64-dbc4-4a98-88a0-b94062d07394 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:03.039926 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07ec64-dbc4-4a98-88a0-b94062d07394 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:03.040195 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07ec64-dbc4-4a98-88a0-b94062d07394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:03.111763 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:03.114365 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-1d07ec64-dbc4-4a98-88a0-b94062d07394 None None] MFA Rules not processed for user `0b3d512304254ffcaa8dad90abf1f9c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:59:03.116135 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-1d07ec64-dbc4-4a98-88a0-b94062d07394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:03.231821 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 545/1090] 10.222.0.13 () {58 vars in 1015 bytes} [Sun Oct 23 19:59:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 19:59:05.770328 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a3f91-ed68-4a52-ac75-43c6e7645439 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:05.770890 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a3f91-ed68-4a52-ac75-43c6e7645439 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:05.770890 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a3f91-ed68-4a52-ac75-43c6e7645439 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:05.772237 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 546/1091] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 19:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:59:05.777695 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f7bd45-2f5c-4c63-b962-daaad0635c42 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:05.778085 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f7bd45-2f5c-4c63-b962-daaad0635c42 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:05.778272 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f7bd45-2f5c-4c63-b962-daaad0635c42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:05.844141 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-98f7bd45-2f5c-4c63-b962-daaad0635c42 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:59:05.845339 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-98f7bd45-2f5c-4c63-b962-daaad0635c42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:05.899560 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:05.941600 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 546/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 19:59:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:59:06.231210 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a3f91-ed68-4a52-ac75-43c6e7645439 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:59:06.232889 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-201a3f91-ed68-4a52-ac75-43c6e7645439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:06.378954 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a3f91-ed68-4a52-ac75-43c6e7645439 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:59:06.451742 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:06.493201 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7706c4a-9bc0-4eb7-ae8e-e6cb76fcb9da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:59:06.493804 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7706c4a-9bc0-4eb7-ae8e-e6cb76fcb9da service nova] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:06.494086 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7706c4a-9bc0-4eb7-ae8e-e6cb76fcb9da service nova] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:06.494397 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7706c4a-9bc0-4eb7-ae8e-e6cb76fcb9da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:06.496001 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-a7706c4a-9bc0-4eb7-ae8e-e6cb76fcb9da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:06.586807 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7706c4a-9bc0-4eb7-ae8e-e6cb76fcb9da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:59:06.588395 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7706c4a-9bc0-4eb7-ae8e-e6cb76fcb9da service nova] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:59:06.606998 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 547/1093] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:59:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:59:07.727813 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec218c7-1d7a-4063-9515-5d4aca34448b None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:07.728376 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec218c7-1d7a-4063-9515-5d4aca34448b None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:07.728376 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec218c7-1d7a-4063-9515-5d4aca34448b None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:07.729746 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 547/1094] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 19:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:59:07.734582 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e3cd8-4feb-41c8-8b58-f73c95d94e66 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:07.734990 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e3cd8-4feb-41c8-8b58-f73c95d94e66 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:07.735137 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93e3cd8-4feb-41c8-8b58-f73c95d94e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:07.809393 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-a93e3cd8-4feb-41c8-8b58-f73c95d94e66 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:59:07.810621 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-a93e3cd8-4feb-41c8-8b58-f73c95d94e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:07.837496 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:07.907009 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 548/1095] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 19:59:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:59:08.056354 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec218c7-1d7a-4063-9515-5d4aca34448b None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:59:08.058433 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7ec218c7-1d7a-4063-9515-5d4aca34448b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:08.202513 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec218c7-1d7a-4063-9515-5d4aca34448b None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:59:08.240854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:08.313308 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40643831-9540-4a47-81bb-283706252aa8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:59:08.313870 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40643831-9540-4a47-81bb-283706252aa8 service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:08.314147 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40643831-9540-4a47-81bb-283706252aa8 service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:08.314461 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40643831-9540-4a47-81bb-283706252aa8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:08.316062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-40643831-9540-4a47-81bb-283706252aa8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:08.404660 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40643831-9540-4a47-81bb-283706252aa8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:59:08.406237 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40643831-9540-4a47-81bb-283706252aa8 service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:59:08.424534 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 548/1096] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:59:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:59:09.534183 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57edc18b-c990-4127-855b-67b51adcb35c None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:09.534780 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57edc18b-c990-4127-855b-67b51adcb35c None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:09.534780 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57edc18b-c990-4127-855b-67b51adcb35c None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:09.536139 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 549/1097] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 19:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:59:09.541142 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfac2fa-3dc1-4872-a088-9c1fca167f21 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:09.541590 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfac2fa-3dc1-4872-a088-9c1fca167f21 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:09.541702 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfac2fa-3dc1-4872-a088-9c1fca167f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:09.595546 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:09.618052 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-8dfac2fa-3dc1-4872-a088-9c1fca167f21 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:59:09.619283 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-8dfac2fa-3dc1-4872-a088-9c1fca167f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:09.713620 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 549/1098] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 19:59:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:59:09.866614 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57edc18b-c990-4127-855b-67b51adcb35c None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:59:09.868173 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-57edc18b-c990-4127-855b-67b51adcb35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:10.013281 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57edc18b-c990-4127-855b-67b51adcb35c None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:59:10.025885 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:10.125790 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d879fbb-bedd-4fcb-ae0c-3331dac6536b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:59:10.126345 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d879fbb-bedd-4fcb-ae0c-3331dac6536b service nova] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:10.126630 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d879fbb-bedd-4fcb-ae0c-3331dac6536b service nova] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:10.126900 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d879fbb-bedd-4fcb-ae0c-3331dac6536b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:10.128484 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-9d879fbb-bedd-4fcb-ae0c-3331dac6536b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:10.217307 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d879fbb-bedd-4fcb-ae0c-3331dac6536b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:59:10.219038 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d879fbb-bedd-4fcb-ae0c-3331dac6536b service nova] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:59:10.236765 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 550/1099] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:59:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:59:11.324597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061ac6b-dfb4-4eec-945b-473ed1352dc0 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:11.325119 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061ac6b-dfb4-4eec-945b-473ed1352dc0 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:11.325216 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4061ac6b-dfb4-4eec-945b-473ed1352dc0 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:11.326613 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 550/1100] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 19:59:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:59:11.331378 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26987b75-a6a3-4064-b25a-10fe25179e5a None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:11.331842 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26987b75-a6a3-4064-b25a-10fe25179e5a None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:11.331974 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26987b75-a6a3-4064-b25a-10fe25179e5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:11.351934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:11.407164 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-26987b75-a6a3-4064-b25a-10fe25179e5a None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:59:11.408367 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-26987b75-a6a3-4064-b25a-10fe25179e5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:11.504703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 551/1101] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 19:59:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 19:59:11.654823 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061ac6b-dfb4-4eec-945b-473ed1352dc0 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:59:11.656271 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4061ac6b-dfb4-4eec-945b-473ed1352dc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:11.766687 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:11.804853 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4061ac6b-dfb4-4eec-945b-473ed1352dc0 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:59:11.915553 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a444826f-b0b3-4662-8662-a74b5c8a566d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:59:11.916079 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444826f-b0b3-4662-8662-a74b5c8a566d service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:11.916360 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444826f-b0b3-4662-8662-a74b5c8a566d service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:11.916630 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a444826f-b0b3-4662-8662-a74b5c8a566d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:11.918853 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a444826f-b0b3-4662-8662-a74b5c8a566d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:11.993056 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:12.006612 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a444826f-b0b3-4662-8662-a74b5c8a566d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:59:12.008227 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a444826f-b0b3-4662-8662-a74b5c8a566d service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:59:12.027617 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 551/1102] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:59:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:59:13.112149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a88ef-f6bf-4f9e-b10d-f8a9f9dd4654 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:13.112685 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a88ef-f6bf-4f9e-b10d-f8a9f9dd4654 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:13.112685 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5a88ef-f6bf-4f9e-b10d-f8a9f9dd4654 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:13.114054 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 552/1103] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 19:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:59:13.118849 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7d636-5eed-48e7-ab71-6c6c234c50d0 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:13.119246 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7d636-5eed-48e7-ab71-6c6c234c50d0 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:13.119394 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7d636-5eed-48e7-ab71-6c6c234c50d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:13.195548 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-a5c7d636-5eed-48e7-ab71-6c6c234c50d0 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:59:13.196760 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a5c7d636-5eed-48e7-ab71-6c6c234c50d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:13.292467 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 552/1104] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 19:59:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:59:13.439000 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a88ef-f6bf-4f9e-b10d-f8a9f9dd4654 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:59:13.440509 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-5a5a88ef-f6bf-4f9e-b10d-f8a9f9dd4654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:13.524685 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:13.585589 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5a88ef-f6bf-4f9e-b10d-f8a9f9dd4654 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:59:13.697214 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b136a0bc-13af-495f-b39c-91022428de35 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:59:13.697815 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136a0bc-13af-495f-b39c-91022428de35 service nova] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:13.698098 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136a0bc-13af-495f-b39c-91022428de35 service nova] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:13.698401 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b136a0bc-13af-495f-b39c-91022428de35 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:13.699992 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-b136a0bc-13af-495f-b39c-91022428de35 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:13.747744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:13.789382 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b136a0bc-13af-495f-b39c-91022428de35 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:59:13.791079 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b136a0bc-13af-495f-b39c-91022428de35 service nova] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:59:13.808689 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 553/1105] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:59:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:59:14.838075 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c2bc1-4ecf-49df-bfd8-5cf96e4a5073 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:14.838620 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c2bc1-4ecf-49df-bfd8-5cf96e4a5073 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:14.838740 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433c2bc1-4ecf-49df-bfd8-5cf96e4a5073 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:14.840011 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 553/1106] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 19:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:59:14.844810 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae607e-ecca-4b86-82bd-8475568cd75a None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:14.845232 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae607e-ecca-4b86-82bd-8475568cd75a None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:14.845344 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae607e-ecca-4b86-82bd-8475568cd75a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:14.919560 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-13ae607e-ecca-4b86-82bd-8475568cd75a None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:59:14.920760 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-13ae607e-ecca-4b86-82bd-8475568cd75a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:15.018044 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 554/1107] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 19:59:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:59:15.169119 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c2bc1-4ecf-49df-bfd8-5cf96e4a5073 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:59:15.170641 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-433c2bc1-4ecf-49df-bfd8-5cf96e4a5073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:15.220247 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:15.317615 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433c2bc1-4ecf-49df-bfd8-5cf96e4a5073 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:59:15.431852 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84627151-73fc-4262-9084-1592b76793ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:59:15.432485 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84627151-73fc-4262-9084-1592b76793ec service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:15.432768 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84627151-73fc-4262-9084-1592b76793ec service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:15.433041 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84627151-73fc-4262-9084-1592b76793ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:15.434669 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-84627151-73fc-4262-9084-1592b76793ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:15.441516 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:15.523396 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84627151-73fc-4262-9084-1592b76793ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:59:15.524979 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84627151-73fc-4262-9084-1592b76793ec service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:59:15.543553 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 554/1108] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:59:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:59:16.574336 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ec0c3-4925-4925-b7e4-cc0439325eb6 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:16.574938 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ec0c3-4925-4925-b7e4-cc0439325eb6 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:16.574938 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ec0c3-4925-4925-b7e4-cc0439325eb6 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:16.576241 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 555/1109] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 19:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:59:16.580970 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b5c6f-ef86-46e4-9351-27023296b5e6 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:16.581417 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b5c6f-ef86-46e4-9351-27023296b5e6 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:16.581558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42b5c6f-ef86-46e4-9351-27023296b5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:16.659170 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-b42b5c6f-ef86-46e4-9351-27023296b5e6 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:59:16.660380 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-b42b5c6f-ef86-46e4-9351-27023296b5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:16.731820 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:16.761689 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 555/1110] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 19:59:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:59:16.909998 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ec0c3-4925-4925-b7e4-cc0439325eb6 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:59:16.911526 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-1f3ec0c3-4925-4925-b7e4-cc0439325eb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:16.929380 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:17.056159 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ec0c3-4925-4925-b7e4-cc0439325eb6 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:59:17.146110 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:17.172098 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fc6083-e5d7-4c66-a0b2-facea3d0d4e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:59:17.172640 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc6083-e5d7-4c66-a0b2-facea3d0d4e1 service nova] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:17.172918 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc6083-e5d7-4c66-a0b2-facea3d0d4e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:17.173189 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fc6083-e5d7-4c66-a0b2-facea3d0d4e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:17.174810 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-86fc6083-e5d7-4c66-a0b2-facea3d0d4e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:17.263255 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fc6083-e5d7-4c66-a0b2-facea3d0d4e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:59:17.264829 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fc6083-e5d7-4c66-a0b2-facea3d0d4e1 service nova] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:59:17.282294 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 556/1111] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:59:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:59:18.312916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bce765-de44-412c-90c4-4f521175133d None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:18.313456 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bce765-de44-412c-90c4-4f521175133d None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:18.313456 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bce765-de44-412c-90c4-4f521175133d None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:18.314779 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 556/1112] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 19:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:59:18.319842 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239cf72-0b79-4a61-bf94-66a315e1004e None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:18.320283 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239cf72-0b79-4a61-bf94-66a315e1004e None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:18.320397 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239cf72-0b79-4a61-bf94-66a315e1004e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:18.398188 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-7239cf72-0b79-4a61-bf94-66a315e1004e None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:59:18.399412 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-7239cf72-0b79-4a61-bf94-66a315e1004e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:18.440955 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:18.494602 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 557/1113] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 19:59:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:59:18.649345 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bce765-de44-412c-90c4-4f521175133d None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:59:18.650911 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-62bce765-de44-412c-90c4-4f521175133d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:18.795430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bce765-de44-412c-90c4-4f521175133d None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:59:18.849552 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:18.907949 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfd87ea-80ec-4e57-840a-7a9f3870bd39 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:59:18.908484 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd87ea-80ec-4e57-840a-7a9f3870bd39 service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:18.908763 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd87ea-80ec-4e57-840a-7a9f3870bd39 service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:18.909035 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfd87ea-80ec-4e57-840a-7a9f3870bd39 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:18.910649 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7dfd87ea-80ec-4e57-840a-7a9f3870bd39 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:18.998886 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfd87ea-80ec-4e57-840a-7a9f3870bd39 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:59:19.000461 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfd87ea-80ec-4e57-840a-7a9f3870bd39 service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:59:19.018208 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 557/1114] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:59:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:59:20.047003 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c76dde-6aaa-42eb-aaca-58e36bf1b81f None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:20.047486 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c76dde-6aaa-42eb-aaca-58e36bf1b81f None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:20.047614 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c76dde-6aaa-42eb-aaca-58e36bf1b81f None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:20.048938 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 558/1115] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 19:59:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:59:20.053735 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6b391-2421-46ce-bbe4-1a47de505af4 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:20.054191 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6b391-2421-46ce-bbe4-1a47de505af4 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:20.054323 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6b391-2421-46ce-bbe4-1a47de505af4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:20.130947 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-2ab6b391-2421-46ce-bbe4-1a47de505af4 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:59:20.132207 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2ab6b391-2421-46ce-bbe4-1a47de505af4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:20.134224 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:20.228804 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 558/1116] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 19:59:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:59:20.378870 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c76dde-6aaa-42eb-aaca-58e36bf1b81f None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:59:20.380411 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-88c76dde-6aaa-42eb-aaca-58e36bf1b81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:20.523539 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c76dde-6aaa-42eb-aaca-58e36bf1b81f None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:59:20.550261 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:20.638812 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b72b534-b192-4aca-bad3-80a9cd1cad37 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:59:20.639332 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b72b534-b192-4aca-bad3-80a9cd1cad37 service nova] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:20.639612 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b72b534-b192-4aca-bad3-80a9cd1cad37 service nova] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:20.639881 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b72b534-b192-4aca-bad3-80a9cd1cad37 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:20.641474 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-2b72b534-b192-4aca-bad3-80a9cd1cad37 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:20.730782 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b72b534-b192-4aca-bad3-80a9cd1cad37 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:59:20.732358 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b72b534-b192-4aca-bad3-80a9cd1cad37 service nova] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:59:20.750196 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 559/1117] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:59:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:59:22.780039 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb178c-9937-465c-80ce-078575a44f56 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:22.780463 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb178c-9937-465c-80ce-078575a44f56 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:22.780589 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb178c-9937-465c-80ce-078575a44f56 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:22.781939 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 559/1118] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 19:59:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:59:22.787196 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7eb778-b7bc-4076-865f-9630625392d8 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:22.787628 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7eb778-b7bc-4076-865f-9630625392d8 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:22.787779 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7eb778-b7bc-4076-865f-9630625392d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:22.828789 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:22.855592 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-6d7eb778-b7bc-4076-865f-9630625392d8 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:59:22.856785 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-6d7eb778-b7bc-4076-865f-9630625392d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:22.952213 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 560/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 19:59:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:59:23.235601 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb178c-9937-465c-80ce-078575a44f56 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:59:23.237119 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-64fb178c-9937-465c-80ce-078575a44f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:23.337654 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb178c-9937-465c-80ce-078575a44f56 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:59:23.369266 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:23.410306 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d7fb9a-b7c3-4533-b224-7b5a0bc8d9e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:59:23.410906 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7fb9a-b7c3-4533-b224-7b5a0bc8d9e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:23.411179 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7fb9a-b7c3-4533-b224-7b5a0bc8d9e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:23.411445 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d7fb9a-b7c3-4533-b224-7b5a0bc8d9e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:23.413006 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-26d7fb9a-b7c3-4533-b224-7b5a0bc8d9e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:23.518234 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d7fb9a-b7c3-4533-b224-7b5a0bc8d9e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:59:23.519672 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d7fb9a-b7c3-4533-b224-7b5a0bc8d9e9 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:59:23.537494 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 560/1120] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:59:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:59:25.834378 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa595a-fc1a-4e37-b39d-2262e786123e None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:25.835003 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa595a-fc1a-4e37-b39d-2262e786123e None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:25.835003 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffa595a-fc1a-4e37-b39d-2262e786123e None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:25.836296 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 561/1121] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 19:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:59:25.841764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c97bcc-ef36-4e1a-ba85-3d2bab6efba8 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:25.842206 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c97bcc-ef36-4e1a-ba85-3d2bab6efba8 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:25.842368 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c97bcc-ef36-4e1a-ba85-3d2bab6efba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:25.908107 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-c9c97bcc-ef36-4e1a-ba85-3d2bab6efba8 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:59:25.909306 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c9c97bcc-ef36-4e1a-ba85-3d2bab6efba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:25.922576 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:26.007129 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 561/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 19:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:59:26.293246 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa595a-fc1a-4e37-b39d-2262e786123e None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:59:26.294835 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-7ffa595a-fc1a-4e37-b39d-2262e786123e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:26.397457 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffa595a-fc1a-4e37-b39d-2262e786123e None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:59:26.410612 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:26.471661 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6be48a-096d-45f1-ac70-7b83bc2abe14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:59:26.472264 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6be48a-096d-45f1-ac70-7b83bc2abe14 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:26.472542 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6be48a-096d-45f1-ac70-7b83bc2abe14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:26.472812 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6be48a-096d-45f1-ac70-7b83bc2abe14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:26.474436 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-7d6be48a-096d-45f1-ac70-7b83bc2abe14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:26.583256 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6be48a-096d-45f1-ac70-7b83bc2abe14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:59:26.584680 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6be48a-096d-45f1-ac70-7b83bc2abe14 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:59:26.603103 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 562/1123] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:59:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 19:59:29.015270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e015373-e2ba-4758-b1ff-4cc3585e2f14 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:29.015802 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e015373-e2ba-4758-b1ff-4cc3585e2f14 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:29.015802 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e015373-e2ba-4758-b1ff-4cc3585e2f14 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:29.017157 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 562/1124] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 19:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:59:29.022474 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb3936-2242-4bf7-b674-01623766e31b None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:29.022847 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb3936-2242-4bf7-b674-01623766e31b None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:29.023023 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bb3936-2242-4bf7-b674-01623766e31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:29.070773 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:29.088985 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-f8bb3936-2242-4bf7-b674-01623766e31b None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:59:29.090226 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-f8bb3936-2242-4bf7-b674-01623766e31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:29.184659 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 563/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 19:59:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:59:29.465435 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e015373-e2ba-4758-b1ff-4cc3585e2f14 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:59:29.466975 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9e015373-e2ba-4758-b1ff-4cc3585e2f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:29.567889 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e015373-e2ba-4758-b1ff-4cc3585e2f14 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:59:29.608861 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:29.642529 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d00da7b-ac90-4aa1-8fe3-9a1c1bbf98ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:59:29.643127 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00da7b-ac90-4aa1-8fe3-9a1c1bbf98ce service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:29.643402 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00da7b-ac90-4aa1-8fe3-9a1c1bbf98ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:29.643673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d00da7b-ac90-4aa1-8fe3-9a1c1bbf98ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:29.645244 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5d00da7b-ac90-4aa1-8fe3-9a1c1bbf98ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:29.752863 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d00da7b-ac90-4aa1-8fe3-9a1c1bbf98ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:59:29.754208 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d00da7b-ac90-4aa1-8fe3-9a1c1bbf98ce service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:59:29.773917 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 563/1126] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:59:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:59:32.052273 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ae13c-1ccb-4ce4-80ee-7e8d9ed6747a None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:32.052733 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ae13c-1ccb-4ce4-80ee-7e8d9ed6747a None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:32.052850 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ae13c-1ccb-4ce4-80ee-7e8d9ed6747a None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:32.054205 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 564/1127] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 19:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:59:32.060113 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93bde3f-2069-4003-bf97-f048085cb3bd None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:32.060507 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93bde3f-2069-4003-bf97-f048085cb3bd None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:32.060655 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93bde3f-2069-4003-bf97-f048085cb3bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:32.125727 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-a93bde3f-2069-4003-bf97-f048085cb3bd None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:59:32.126938 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a93bde3f-2069-4003-bf97-f048085cb3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:32.147591 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:32.225221 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 564/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 19:59:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:59:32.512927 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ae13c-1ccb-4ce4-80ee-7e8d9ed6747a None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:59:32.514504 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-492ae13c-1ccb-4ce4-80ee-7e8d9ed6747a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:32.620695 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ae13c-1ccb-4ce4-80ee-7e8d9ed6747a None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:59:32.642396 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:32.696799 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf296f-6517-4ee8-a247-ee959b925d93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:59:32.697402 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf296f-6517-4ee8-a247-ee959b925d93 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:32.697678 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf296f-6517-4ee8-a247-ee959b925d93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:32.698585 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf296f-6517-4ee8-a247-ee959b925d93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:32.700154 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-43bf296f-6517-4ee8-a247-ee959b925d93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:32.809447 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf296f-6517-4ee8-a247-ee959b925d93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:59:32.810906 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bf296f-6517-4ee8-a247-ee959b925d93 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:59:32.829243 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 565/1129] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:59:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:59:35.011315 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c89c7-d6c7-4299-bac1-9a782587d7ea None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:35.011842 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c89c7-d6c7-4299-bac1-9a782587d7ea None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:35.011842 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c89c7-d6c7-4299-bac1-9a782587d7ea None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:35.013224 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 565/1130] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 19:59:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:59:35.018900 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92317412-7cd4-4bab-806a-0203b51f5a8f None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:35.019343 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92317412-7cd4-4bab-806a-0203b51f5a8f None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:35.019455 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92317412-7cd4-4bab-806a-0203b51f5a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:35.083707 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:35.086658 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-92317412-7cd4-4bab-806a-0203b51f5a8f None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:59:35.088493 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-92317412-7cd4-4bab-806a-0203b51f5a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:35.185051 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 566/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 19:59:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:59:35.470707 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6c89c7-d6c7-4299-bac1-9a782587d7ea None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:59:35.472228 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7f6c89c7-d6c7-4299-bac1-9a782587d7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:35.572361 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6c89c7-d6c7-4299-bac1-9a782587d7ea None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:59:35.620460 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:35.644788 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290348bc-5bb6-4a38-afbf-8cc5d6bb6068 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:59:35.645384 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290348bc-5bb6-4a38-afbf-8cc5d6bb6068 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:35.645656 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290348bc-5bb6-4a38-afbf-8cc5d6bb6068 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:35.645921 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290348bc-5bb6-4a38-afbf-8cc5d6bb6068 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:35.647518 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-290348bc-5bb6-4a38-afbf-8cc5d6bb6068 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:35.754456 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290348bc-5bb6-4a38-afbf-8cc5d6bb6068 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:59:35.755903 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290348bc-5bb6-4a38-afbf-8cc5d6bb6068 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:59:35.774931 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 566/1132] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:59:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:59:37.952087 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee0351-1cf7-48e2-b5b3-e34097218f9c None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:37.952709 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee0351-1cf7-48e2-b5b3-e34097218f9c None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:37.952709 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee0351-1cf7-48e2-b5b3-e34097218f9c None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:37.953977 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 567/1133] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 19:59:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:59:37.959284 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874dfc3e-330e-4988-a8a3-84594eaabf6a None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:37.959684 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874dfc3e-330e-4988-a8a3-84594eaabf6a None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:37.959831 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874dfc3e-330e-4988-a8a3-84594eaabf6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:38.027115 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-874dfc3e-330e-4988-a8a3-84594eaabf6a None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:59:38.028311 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-874dfc3e-330e-4988-a8a3-84594eaabf6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:38.057004 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:38.124628 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 567/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 19:59:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:59:38.407306 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee0351-1cf7-48e2-b5b3-e34097218f9c None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:59:38.408857 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-0dee0351-1cf7-48e2-b5b3-e34097218f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:38.509481 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dee0351-1cf7-48e2-b5b3-e34097218f9c None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:59:38.537870 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:38.581863 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e619d869-bcf2-4721-8348-cf378a6fe845 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:59:38.582487 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619d869-bcf2-4721-8348-cf378a6fe845 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:38.582768 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619d869-bcf2-4721-8348-cf378a6fe845 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:38.583046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e619d869-bcf2-4721-8348-cf378a6fe845 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:38.584626 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-e619d869-bcf2-4721-8348-cf378a6fe845 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:38.691854 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e619d869-bcf2-4721-8348-cf378a6fe845 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:59:38.693188 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e619d869-bcf2-4721-8348-cf378a6fe845 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:59:38.711691 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 568/1135] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:59:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:59:40.897923 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370c7a8-0587-4a85-a4ee-5433787500f9 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:40.898527 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370c7a8-0587-4a85-a4ee-5433787500f9 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:40.898527 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370c7a8-0587-4a85-a4ee-5433787500f9 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:40.899899 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 568/1136] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 19:59:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:59:40.905632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7920bb-609e-4dc5-b299-a3e6f730578d None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:40.906053 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7920bb-609e-4dc5-b299-a3e6f730578d None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:40.906180 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7920bb-609e-4dc5-b299-a3e6f730578d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:40.971347 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-7a7920bb-609e-4dc5-b299-a3e6f730578d None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:59:40.972547 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-7a7920bb-609e-4dc5-b299-a3e6f730578d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:40.980684 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:41.068662 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 569/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 19:59:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:59:41.347546 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d370c7a8-0587-4a85-a4ee-5433787500f9 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:59:41.349130 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d370c7a8-0587-4a85-a4ee-5433787500f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:41.452291 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d370c7a8-0587-4a85-a4ee-5433787500f9 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:59:41.507166 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:41.525199 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a38fc-0b6f-421e-b51b-faf3bca1298e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:59:41.525820 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a38fc-0b6f-421e-b51b-faf3bca1298e service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:41.526091 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a38fc-0b6f-421e-b51b-faf3bca1298e service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:41.526381 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a38fc-0b6f-421e-b51b-faf3bca1298e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:41.527947 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c35a38fc-0b6f-421e-b51b-faf3bca1298e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:41.634662 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35a38fc-0b6f-421e-b51b-faf3bca1298e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:59:41.636004 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35a38fc-0b6f-421e-b51b-faf3bca1298e service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:59:41.653963 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 569/1138] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:59:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:59:43.816179 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd1e48-0153-4444-8764-f5fdaffcc4cc None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:43.816747 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd1e48-0153-4444-8764-f5fdaffcc4cc None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:43.816747 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd1e48-0153-4444-8764-f5fdaffcc4cc None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:43.818118 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 570/1139] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 19:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:59:43.824343 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc41df-d66d-4424-b3fb-d4dd4e6f116e None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:43.824759 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc41df-d66d-4424-b3fb-d4dd4e6f116e None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:43.824894 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc41df-d66d-4424-b3fb-d4dd4e6f116e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:43.890035 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-cdcc41df-d66d-4424-b3fb-d4dd4e6f116e None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:59:43.891265 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-cdcc41df-d66d-4424-b3fb-d4dd4e6f116e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:43.928816 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:43.987503 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 570/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 19:59:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 19:59:44.272465 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd1e48-0153-4444-8764-f5fdaffcc4cc None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:59:44.274027 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-4edd1e48-0153-4444-8764-f5fdaffcc4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:44.377692 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd1e48-0153-4444-8764-f5fdaffcc4cc None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:59:44.413973 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:44.449975 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c54262f-0045-43c2-b316-7afb4bee4a1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:59:44.450606 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54262f-0045-43c2-b316-7afb4bee4a1d service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:44.450880 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54262f-0045-43c2-b316-7afb4bee4a1d service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:44.451148 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54262f-0045-43c2-b316-7afb4bee4a1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:44.452718 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-4c54262f-0045-43c2-b316-7afb4bee4a1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:44.559918 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c54262f-0045-43c2-b316-7afb4bee4a1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:59:44.561385 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c54262f-0045-43c2-b316-7afb4bee4a1d service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:59:44.579263 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 571/1141] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:59:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:59:46.732742 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5d2b3-1a52-4ecd-9e8a-ed9fa20fe649 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:46.733295 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5d2b3-1a52-4ecd-9e8a-ed9fa20fe649 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:46.733525 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d5d2b3-1a52-4ecd-9e8a-ed9fa20fe649 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:46.735821 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 571/1142] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 19:59:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:59:46.741783 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c70b2-5c42-42a8-9ca9-33f19e25c1d7 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:46.742224 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c70b2-5c42-42a8-9ca9-33f19e25c1d7 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:46.742380 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c70b2-5c42-42a8-9ca9-33f19e25c1d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:46.809569 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-4e9c70b2-5c42-42a8-9ca9-33f19e25c1d7 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:59:46.810807 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-4e9c70b2-5c42-42a8-9ca9-33f19e25c1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:46.828607 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:46.907692 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 572/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 19:59:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:59:47.189994 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d5d2b3-1a52-4ecd-9e8a-ed9fa20fe649 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:59:47.191619 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-b3d5d2b3-1a52-4ecd-9e8a-ed9fa20fe649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:47.295210 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d5d2b3-1a52-4ecd-9e8a-ed9fa20fe649 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:59:47.362378 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:47.369484 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9b22a5-b015-46cc-8cf8-acb06bbdc88e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:59:47.370090 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b22a5-b015-46cc-8cf8-acb06bbdc88e service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:47.370401 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b22a5-b015-46cc-8cf8-acb06bbdc88e service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:47.370704 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9b22a5-b015-46cc-8cf8-acb06bbdc88e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:47.372337 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-dc9b22a5-b015-46cc-8cf8-acb06bbdc88e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:47.479836 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9b22a5-b015-46cc-8cf8-acb06bbdc88e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:59:47.481260 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9b22a5-b015-46cc-8cf8-acb06bbdc88e service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:59:47.498636 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 572/1144] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:59:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:59:49.626103 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79076d33-40d2-4fca-aaf1-c100c8a2a406 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:49.626667 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79076d33-40d2-4fca-aaf1-c100c8a2a406 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:49.626785 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79076d33-40d2-4fca-aaf1-c100c8a2a406 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:49.628062 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 573/1145] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 19:59:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 19:59:49.633478 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c4c05-434d-4983-b520-cfa4223f6062 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:49.633895 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c4c05-434d-4983-b520-cfa4223f6062 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:49.634036 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8c4c05-434d-4983-b520-cfa4223f6062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:49.699936 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-7a8c4c05-434d-4983-b520-cfa4223f6062 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 19:59:49.701136 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7a8c4c05-434d-4983-b520-cfa4223f6062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:49.748929 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:49.798608 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 573/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 19:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 19:59:50.081547 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79076d33-40d2-4fca-aaf1-c100c8a2a406 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 19:59:50.083168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-79076d33-40d2-4fca-aaf1-c100c8a2a406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:50.185513 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79076d33-40d2-4fca-aaf1-c100c8a2a406 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 19:59:50.234770 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 19:59:50.260906 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb07fa0-4888-46ea-ab40-907de7f1f102 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 19:59:50.261509 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb07fa0-4888-46ea-ab40-907de7f1f102 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 19:59:50.261793 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb07fa0-4888-46ea-ab40-907de7f1f102 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 19:59:50.262065 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb07fa0-4888-46ea-ab40-907de7f1f102 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 19:59:50.263696 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-6fb07fa0-4888-46ea-ab40-907de7f1f102 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 19:59:50.370993 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb07fa0-4888-46ea-ab40-907de7f1f102 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 19:59:50.372412 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb07fa0-4888-46ea-ab40-907de7f1f102 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 19:59:50.390353 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 574/1147] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 19:59:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 19:59:59.797043 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=22 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:00:00.388208 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=66 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:00:10.122423 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddbfe3a-cebb-40a0-ba8b-4bf2ca2966f8 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:00:10.122965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddbfe3a-cebb-40a0-ba8b-4bf2ca2966f8 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:00:10.122965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddbfe3a-cebb-40a0-ba8b-4bf2ca2966f8 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:00:10.124341 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 574/1148] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 20:00:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:00:10.129403 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790a9af-5c86-4701-8ffe-da84fc68b18a None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:00:10.129834 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790a9af-5c86-4701-8ffe-da84fc68b18a None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:00:10.129973 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d790a9af-5c86-4701-8ffe-da84fc68b18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:00:10.196915 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-d790a9af-5c86-4701-8ffe-da84fc68b18a None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:00:10.198114 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-d790a9af-5c86-4701-8ffe-da84fc68b18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:00:10.292882 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 575/1149] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 20:00:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:00:10.589867 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeca072-80f9-4e87-a288-586b6d30d260 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:00:10.590148 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeca072-80f9-4e87-a288-586b6d30d260 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:00:10.590451 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eeca072-80f9-4e87-a288-586b6d30d260 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:00:10.591778 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 575/1150] 10.222.0.13 () {54 vars in 962 bytes} [Sun Oct 23 20:00:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:00:10.596765 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd79087-81b4-4467-b67d-f70fdb074f95 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:00:10.597055 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd79087-81b4-4467-b67d-f70fdb074f95 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:00:10.597333 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd79087-81b4-4467-b67d-f70fdb074f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:00:10.671864 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-8fd79087-81b4-4467-b67d-f70fdb074f95 None None] MFA Rules not processed for user `021216f90d7f4ff6b6eb7a1d86399d99`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:00:10.673046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-8fd79087-81b4-4467-b67d-f70fdb074f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:00:10.692728 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:00:10.748394 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 576/1151] 10.222.0.13 () {58 vars in 1059 bytes} [Sun Oct 23 20:00:10 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:00:10.756075 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeca072-80f9-4e87-a288-586b6d30d260 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:00:10.757547 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-1eeca072-80f9-4e87-a288-586b6d30d260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:00:10.860619 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eeca072-80f9-4e87-a288-586b6d30d260 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:00:10.935876 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64078b37-c124-4013-9caf-572dea6eeed9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '021216f90d7f4ff6b6eb7a1d86399d99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:00:10.936476 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64078b37-c124-4013-9caf-572dea6eeed9 service glance] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:00:10.936753 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64078b37-c124-4013-9caf-572dea6eeed9 service glance] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:00:10.937022 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64078b37-c124-4013-9caf-572dea6eeed9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:00:10.938639 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-64078b37-c124-4013-9caf-572dea6eeed9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:00:10.992106 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:00:11.048026 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64078b37-c124-4013-9caf-572dea6eeed9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:00:11.049459 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64078b37-c124-4013-9caf-572dea6eeed9 service glance] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:00:11.068636 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 576/1152] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:00:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:00:13.291318 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aaba37-0b24-49fe-90dc-9dcb0fa255cf None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:00:13.291854 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aaba37-0b24-49fe-90dc-9dcb0fa255cf None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:00:13.291854 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aaba37-0b24-49fe-90dc-9dcb0fa255cf None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:00:13.293200 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 577/1153] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 20:00:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:00:13.298553 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fea0dc-59fa-4f58-b778-97fec52f1c22 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:00:13.298976 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fea0dc-59fa-4f58-b778-97fec52f1c22 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:00:13.299084 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fea0dc-59fa-4f58-b778-97fec52f1c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:00:13.364470 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-41fea0dc-59fa-4f58-b778-97fec52f1c22 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:00:13.365654 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-41fea0dc-59fa-4f58-b778-97fec52f1c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:00:13.460423 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 577/1154] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 20:00:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:00:13.732880 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aaba37-0b24-49fe-90dc-9dcb0fa255cf None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:00:13.734448 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-37aaba37-0b24-49fe-90dc-9dcb0fa255cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:00:13.857925 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aaba37-0b24-49fe-90dc-9dcb0fa255cf None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:00:13.906858 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:00:13.950709 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75719bf2-606a-489c-983d-a59af94aec61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:00:13.951306 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75719bf2-606a-489c-983d-a59af94aec61 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:00:13.951586 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75719bf2-606a-489c-983d-a59af94aec61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:00:13.951856 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75719bf2-606a-489c-983d-a59af94aec61 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:00:13.952496 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75719bf2-606a-489c-983d-a59af94aec61 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:00:13.953587 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: WARNING py.warnings [None req-75719bf2-606a-489c-983d-a59af94aec61 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 20:00:13.953587 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: warnings.warn(msg) Oct 23 20:00:13.953587 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: Oct 23 20:00:13.954755 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75719bf2-606a-489c-983d-a59af94aec61 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:00:13.974617 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 578/1155] 10.222.0.13 () {56 vars in 1108 bytes} [Sun Oct 23 20:00:13 2022] GET /identity/v3/projects => generated 2553 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:00:16.145808 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008ec9f-c555-49e3-89f9-f2fa658b7447 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:00:16.146382 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008ec9f-c555-49e3-89f9-f2fa658b7447 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:00:16.146382 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3008ec9f-c555-49e3-89f9-f2fa658b7447 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:00:16.147815 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 578/1156] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 20:00:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:00:16.154233 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4b84d-3eba-4cd1-aa8f-1cb1ea9f7a8c None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:00:16.154687 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4b84d-3eba-4cd1-aa8f-1cb1ea9f7a8c None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:00:16.154803 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd4b84d-3eba-4cd1-aa8f-1cb1ea9f7a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:00:16.220033 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-1dd4b84d-3eba-4cd1-aa8f-1cb1ea9f7a8c None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:00:16.221226 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-1dd4b84d-3eba-4cd1-aa8f-1cb1ea9f7a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:00:16.314102 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 579/1157] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 20:00:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:00:16.609868 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3008ec9f-c555-49e3-89f9-f2fa658b7447 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:00:16.611397 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3008ec9f-c555-49e3-89f9-f2fa658b7447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:00:16.621572 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:00:16.758357 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3008ec9f-c555-49e3-89f9-f2fa658b7447 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:00:16.839070 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:00:16.873212 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fdc14a-41a7-46f1-ba3f-857d1e364c90 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:00:16.873752 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdc14a-41a7-46f1-ba3f-857d1e364c90 service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:00:16.874029 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdc14a-41a7-46f1-ba3f-857d1e364c90 service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:00:16.874332 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fdc14a-41a7-46f1-ba3f-857d1e364c90 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:00:16.875934 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-90fdc14a-41a7-46f1-ba3f-857d1e364c90 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:00:16.965216 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fdc14a-41a7-46f1-ba3f-857d1e364c90 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:00:16.966812 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fdc14a-41a7-46f1-ba3f-857d1e364c90 service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:00:16.984773 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 579/1158] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:00:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:00:19.308759 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03591eac-99dd-4671-a3c8-0e8d09878d49 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:00:19.309375 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03591eac-99dd-4671-a3c8-0e8d09878d49 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:00:19.309375 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03591eac-99dd-4671-a3c8-0e8d09878d49 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:00:19.310729 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 580/1159] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 20:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:00:19.315894 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cebed9a-3f47-41ce-bbee-dcdcb8485d69 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:00:19.316338 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cebed9a-3f47-41ce-bbee-dcdcb8485d69 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:00:19.316485 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cebed9a-3f47-41ce-bbee-dcdcb8485d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:00:19.382563 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-9cebed9a-3f47-41ce-bbee-dcdcb8485d69 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:00:19.383796 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9cebed9a-3f47-41ce-bbee-dcdcb8485d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:00:19.429615 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:00:19.479174 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 580/1160] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 20:00:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:00:19.764538 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03591eac-99dd-4671-a3c8-0e8d09878d49 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:00:19.766416 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-03591eac-99dd-4671-a3c8-0e8d09878d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:00:19.791780 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:00:19.872425 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03591eac-99dd-4671-a3c8-0e8d09878d49 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:00:19.944686 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da01206-36af-4432-a507-257ff0f91229 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:00:19.945287 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da01206-36af-4432-a507-257ff0f91229 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:00:19.945568 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da01206-36af-4432-a507-257ff0f91229 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:00:19.946406 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da01206-36af-4432-a507-257ff0f91229 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:00:19.947992 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-7da01206-36af-4432-a507-257ff0f91229 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:00:20.021278 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:00:20.056362 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da01206-36af-4432-a507-257ff0f91229 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:00:20.058304 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da01206-36af-4432-a507-257ff0f91229 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:00:20.076881 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 581/1161] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:00:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:00:22.277596 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df321f-02af-4b91-8a0c-0b39dd83fde0 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:00:22.278121 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df321f-02af-4b91-8a0c-0b39dd83fde0 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:00:22.278121 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df321f-02af-4b91-8a0c-0b39dd83fde0 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:00:22.279456 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 581/1162] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 20:00:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:00:22.284623 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5df72-a930-4f6d-8839-58953b58fc71 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:00:22.285022 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5df72-a930-4f6d-8839-58953b58fc71 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:00:22.285180 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5df72-a930-4f6d-8839-58953b58fc71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:00:22.350963 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-70e5df72-a930-4f6d-8839-58953b58fc71 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:00:22.352755 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-70e5df72-a930-4f6d-8839-58953b58fc71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:00:22.448896 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 582/1163] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 20:00:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:00:22.737917 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df321f-02af-4b91-8a0c-0b39dd83fde0 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:00:22.739482 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-81df321f-02af-4b91-8a0c-0b39dd83fde0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:00:22.843888 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df321f-02af-4b91-8a0c-0b39dd83fde0 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:00:22.915929 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:00:22.917518 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2110eb32-9ac8-43bb-8a55-8b1b481e8a3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:00:22.918206 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110eb32-9ac8-43bb-8a55-8b1b481e8a3f service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:00:22.918708 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110eb32-9ac8-43bb-8a55-8b1b481e8a3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:00:22.919202 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110eb32-9ac8-43bb-8a55-8b1b481e8a3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:00:22.921126 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2110eb32-9ac8-43bb-8a55-8b1b481e8a3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:00:23.032372 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2110eb32-9ac8-43bb-8a55-8b1b481e8a3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:00:23.033799 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2110eb32-9ac8-43bb-8a55-8b1b481e8a3f service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:00:23.051932 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 582/1164] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:00:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:00:25.339222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342afd57-1b07-4193-855b-c43a278dc9b9 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:00:25.339749 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342afd57-1b07-4193-855b-c43a278dc9b9 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:00:25.339749 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342afd57-1b07-4193-855b-c43a278dc9b9 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:00:25.341045 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 583/1165] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 20:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:00:25.346299 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c977cc-e7a8-42ab-b644-d292ea4bfe01 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:00:25.346752 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c977cc-e7a8-42ab-b644-d292ea4bfe01 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:00:25.346861 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c977cc-e7a8-42ab-b644-d292ea4bfe01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:00:25.412745 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-f0c977cc-e7a8-42ab-b644-d292ea4bfe01 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:00:25.413955 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f0c977cc-e7a8-42ab-b644-d292ea4bfe01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:00:25.466512 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:00:25.510892 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 583/1166] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 20:00:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:00:25.805063 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342afd57-1b07-4193-855b-c43a278dc9b9 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:00:25.806635 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-342afd57-1b07-4193-855b-c43a278dc9b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:00:25.843821 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:00:25.910538 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342afd57-1b07-4193-855b-c43a278dc9b9 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:00:25.983370 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea56959-d47f-4876-98e9-242e9b65e52a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:00:25.983987 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea56959-d47f-4876-98e9-242e9b65e52a service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:00:25.984265 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea56959-d47f-4876-98e9-242e9b65e52a service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:00:25.984535 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea56959-d47f-4876-98e9-242e9b65e52a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:00:25.986121 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-5ea56959-d47f-4876-98e9-242e9b65e52a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:00:26.065709 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:00:26.095779 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea56959-d47f-4876-98e9-242e9b65e52a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:00:26.097247 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea56959-d47f-4876-98e9-242e9b65e52a service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:00:26.115082 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 584/1167] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:00:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:00:28.425701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a1448-d2a1-47aa-887d-98f29466ba8d None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:00:28.426281 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a1448-d2a1-47aa-887d-98f29466ba8d None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:00:28.426281 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192a1448-d2a1-47aa-887d-98f29466ba8d None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:00:28.427738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 584/1168] 10.222.0.13 () {54 vars in 927 bytes} [Sun Oct 23 20:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:00:28.432914 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4133b9-131b-4d6c-a971-2bd8713150e5 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:00:28.433354 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4133b9-131b-4d6c-a971-2bd8713150e5 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:00:28.433461 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4133b9-131b-4d6c-a971-2bd8713150e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:00:28.498540 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-ea4133b9-131b-4d6c-a971-2bd8713150e5 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:00:28.499731 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-ea4133b9-131b-4d6c-a971-2bd8713150e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:00:28.593728 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 585/1169] 10.222.0.13 () {58 vars in 1024 bytes} [Sun Oct 23 20:00:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:00:28.882422 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a1448-d2a1-47aa-887d-98f29466ba8d None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:00:28.884536 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-192a1448-d2a1-47aa-887d-98f29466ba8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:00:28.987312 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192a1448-d2a1-47aa-887d-98f29466ba8d None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:00:29.061421 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fbdc84-a042-43b6-91ad-f0f4bd8b3617 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:00:29.061955 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbdc84-a042-43b6-91ad-f0f4bd8b3617 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:00:29.062230 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbdc84-a042-43b6-91ad-f0f4bd8b3617 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:00:29.062528 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fbdc84-a042-43b6-91ad-f0f4bd8b3617 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:00:29.064101 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f4fbdc84-a042-43b6-91ad-f0f4bd8b3617 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:00:29.073292 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:00:29.172460 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fbdc84-a042-43b6-91ad-f0f4bd8b3617 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:00:29.173889 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fbdc84-a042-43b6-91ad-f0f4bd8b3617 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:00:29.192169 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 585/1170] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:00:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:00:38.592281 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=82 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:00:39.189919 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=50 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:01:48.593106 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe30e64-d920-4f0e-8d0e-5b9347e7d1f2 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:01:48.593697 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe30e64-d920-4f0e-8d0e-5b9347e7d1f2 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:01:48.593697 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe30e64-d920-4f0e-8d0e-5b9347e7d1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:01:48.669026 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-fbe30e64-d920-4f0e-8d0e-5b9347e7d1f2 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:01:48.670228 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-fbe30e64-d920-4f0e-8d0e-5b9347e7d1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:01:48.766609 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 586/1171] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:01:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:01:48.774649 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a971055c-1a44-4550-8b75-87df0ee6e945 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:01:48.775186 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a971055c-1a44-4550-8b75-87df0ee6e945 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:01:48.775296 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a971055c-1a44-4550-8b75-87df0ee6e945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:01:48.850213 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-a971055c-1a44-4550-8b75-87df0ee6e945 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:01:48.851379 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a971055c-1a44-4550-8b75-87df0ee6e945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:01:48.947371 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 586/1172] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:01:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:01:48.952013 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe30e64-d920-4f0e-8d0e-5b9347e7d1f2 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:01:48.953199 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-fbe30e64-d920-4f0e-8d0e-5b9347e7d1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:01:48.964344 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe30e64-d920-4f0e-8d0e-5b9347e7d1f2 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:01:49.025289 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:01:49.059680 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981ebb02-693d-40ca-9d37-195feaad2c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:01:49.060317 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ebb02-693d-40ca-9d37-195feaad2c6a admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:01:49.060597 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ebb02-693d-40ca-9d37-195feaad2c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:01:49.060865 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981ebb02-693d-40ca-9d37-195feaad2c6a admin admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:01:49.061504 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981ebb02-693d-40ca-9d37-195feaad2c6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:01:49.061806 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981ebb02-693d-40ca-9d37-195feaad2c6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:01:49.062843 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: WARNING py.warnings [None req-981ebb02-693d-40ca-9d37-195feaad2c6a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 23 20:01:49.062843 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: warnings.warn(msg) Oct 23 20:01:49.062843 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: Oct 23 20:01:49.063729 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981ebb02-693d-40ca-9d37-195feaad2c6a admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:01:49.075140 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 587/1173] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:01:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:01:49.084446 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a971055c-1a44-4550-8b75-87df0ee6e945 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:01:49.093458 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a971055c-1a44-4550-8b75-87df0ee6e945 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:01:49.154797 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:01:49.188585 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7bddaf-74ff-4505-bf0e-b2954b0f3867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:01:49.189184 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bddaf-74ff-4505-bf0e-b2954b0f3867 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:01:49.189459 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bddaf-74ff-4505-bf0e-b2954b0f3867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:01:49.189727 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7bddaf-74ff-4505-bf0e-b2954b0f3867 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:01:49.190382 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7bddaf-74ff-4505-bf0e-b2954b0f3867 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:01:49.191470 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: WARNING py.warnings [None req-6a7bddaf-74ff-4505-bf0e-b2954b0f3867 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 20:01:49.191470 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: warnings.warn(msg) Oct 23 20:01:49.191470 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: Oct 23 20:01:49.192646 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7bddaf-74ff-4505-bf0e-b2954b0f3867 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:01:49.248032 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 587/1174] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:01:49 2022] POST /identity/v3/projects => generated 379 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:01:49.252232 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981ebb02-693d-40ca-9d37-195feaad2c6a admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:01:49.253720 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-981ebb02-693d-40ca-9d37-195feaad2c6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:01:49.287246 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981ebb02-693d-40ca-9d37-195feaad2c6a admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:01:49.383645 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc29b88-d600-4db9-b603-1c7796510af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:01:49.384280 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc29b88-d600-4db9-b603-1c7796510af4 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:01:49.384583 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc29b88-d600-4db9-b603-1c7796510af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:01:49.384855 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc29b88-d600-4db9-b603-1c7796510af4 admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:01:49.385488 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc29b88-d600-4db9-b603-1c7796510af4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:01:49.386528 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: WARNING py.warnings [None req-acc29b88-d600-4db9-b603-1c7796510af4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 20:01:49.386528 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: warnings.warn(msg) Oct 23 20:01:49.386528 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: Oct 23 20:01:49.387730 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc29b88-d600-4db9-b603-1c7796510af4 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:01:49.431061 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 588/1175] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:01:49 2022] POST /identity/v3/users => generated 290 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:01:49.434844 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7bddaf-74ff-4505-bf0e-b2954b0f3867 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:01:49.436366 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-6a7bddaf-74ff-4505-bf0e-b2954b0f3867 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:01:49.498008 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7bddaf-74ff-4505-bf0e-b2954b0f3867 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:01:49.581887 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:01:49.593175 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5bc03c-6778-4836-b6ec-2f454dd8383f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:01:49.593785 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bc03c-6778-4836-b6ec-2f454dd8383f admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:01:49.594089 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bc03c-6778-4836-b6ec-2f454dd8383f admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:01:49.594414 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5bc03c-6778-4836-b6ec-2f454dd8383f admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:01:49.595053 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5bc03c-6778-4836-b6ec-2f454dd8383f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:01:49.596640 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5bc03c-6778-4836-b6ec-2f454dd8383f admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:01:49.609212 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 588/1176] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:01:49 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:01:49.613338 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc29b88-d600-4db9-b603-1c7796510af4 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:01:49.615524 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-acc29b88-d600-4db9-b603-1c7796510af4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:01:49.626736 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc29b88-d600-4db9-b603-1c7796510af4 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:01:49.650846 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:01:49.720194 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c22b7d-f6b4-4c3f-8378-6c42914280b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:01:49.720717 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c22b7d-f6b4-4c3f-8378-6c42914280b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:01:49.720993 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c22b7d-f6b4-4c3f-8378-6c42914280b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:01:49.721263 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c22b7d-f6b4-4c3f-8378-6c42914280b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:01:49.721904 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c22b7d-f6b4-4c3f-8378-6c42914280b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:01:49.723514 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c22b7d-f6b4-4c3f-8378-6c42914280b4 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:01:49.736195 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 589/1177] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:01:49 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:01:49.740870 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5bc03c-6778-4836-b6ec-2f454dd8383f admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:01:49.742421 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9f5bc03c-6778-4836-b6ec-2f454dd8383f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:01:49.775713 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5bc03c-6778-4836-b6ec-2f454dd8383f admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:01:49.868080 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7b08f-736f-485f-8d9b-693eaf38ca54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:01:49.868728 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7b08f-736f-485f-8d9b-693eaf38ca54 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:01:49.869004 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7b08f-736f-485f-8d9b-693eaf38ca54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:01:49.869273 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7b08f-736f-485f-8d9b-693eaf38ca54 admin admin] PATH_INFO: `/v3/projects/16b8829fed62408bafc50ad32939bb25/users/2b86ea60f033481795115ea5b30bfcb6/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:01:49.891582 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d7b08f-736f-485f-8d9b-693eaf38ca54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16b8829fed62408bafc50ad32939bb25, user_id=2b86ea60f033481795115ea5b30bfcb6, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:01:49.894322 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d7b08f-736f-485f-8d9b-693eaf38ca54 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:01:49.918510 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 589/1178] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:01:49 2022] PUT /identity/v3/projects/16b8829fed62408bafc50ad32939bb25/users/2b86ea60f033481795115ea5b30bfcb6/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:01:49.929817 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc48696-03bb-47e8-a391-4ecb2d67e4f2 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:01:49.930107 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc48696-03bb-47e8-a391-4ecb2d67e4f2 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:01:49.930419 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc48696-03bb-47e8-a391-4ecb2d67e4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:01:49.986707 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-6bc48696-03bb-47e8-a391-4ecb2d67e4f2 None None] MFA Rules not processed for user `2b86ea60f033481795115ea5b30bfcb6`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:01:49.987827 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-6bc48696-03bb-47e8-a391-4ecb2d67e4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:01:50.064716 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 590/1179] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:01:49 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:01:50.078698 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ae68e-af1f-4b38-ac3f-b88e3231f939 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:01:50.078989 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ae68e-af1f-4b38-ac3f-b88e3231f939 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:01:50.079274 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420ae68e-af1f-4b38-ac3f-b88e3231f939 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:01:50.080615 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 590/1180] 10.222.0.13 () {54 vars in 963 bytes} [Sun Oct 23 20:01:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:01:50.085350 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3df69-f12e-48db-8918-6a4fc6f476be None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:01:50.085635 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3df69-f12e-48db-8918-6a4fc6f476be None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:01:50.085910 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3df69-f12e-48db-8918-6a4fc6f476be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:01:50.116773 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:01:50.159942 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-b2b3df69-f12e-48db-8918-6a4fc6f476be None None] MFA Rules not processed for user `7bcc5f5101fb4913aaf5caf564ca6b60`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:01:50.161166 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-b2b3df69-f12e-48db-8918-6a4fc6f476be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:01:50.234125 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 591/1181] 10.222.0.13 () {58 vars in 1060 bytes} [Sun Oct 23 20:01:50 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:01:50.242565 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ae68e-af1f-4b38-ac3f-b88e3231f939 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:01:50.244023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-420ae68e-af1f-4b38-ac3f-b88e3231f939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:01:50.294120 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:01:50.348493 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420ae68e-af1f-4b38-ac3f-b88e3231f939 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:01:50.422434 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e62371-3608-4715-8324-7ca4987ef1df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:01:50.423044 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e62371-3608-4715-8324-7ca4987ef1df service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:01:50.423323 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e62371-3608-4715-8324-7ca4987ef1df service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:01:50.423599 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e62371-3608-4715-8324-7ca4987ef1df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:01:50.425189 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f9e62371-3608-4715-8324-7ca4987ef1df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:01:50.524468 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e62371-3608-4715-8324-7ca4987ef1df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:01:50.525815 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e62371-3608-4715-8324-7ca4987ef1df service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:01:50.529286 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:01:50.544114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 591/1182] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:01:50 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:01:50.589522 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae98ab-d4f7-4c74-ab61-6414a99c1f67 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:01:50.589808 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae98ab-d4f7-4c74-ab61-6414a99c1f67 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:01:50.590080 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae98ab-d4f7-4c74-ab61-6414a99c1f67 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:01:50.591370 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 592/1183] 10.222.0.13 () {52 vars in 830 bytes} [Sun Oct 23 20:01:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:01:50.594871 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e62371-3608-4715-8324-7ca4987ef1df service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:01:50.596074 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f9e62371-3608-4715-8324-7ca4987ef1df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:01:50.694274 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e62371-3608-4715-8324-7ca4987ef1df service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:01:50.786432 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a215bd-9315-4907-bdfa-83a4e214e21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:01:50.787072 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a215bd-9315-4907-bdfa-83a4e214e21e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:01:50.787254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a215bd-9315-4907-bdfa-83a4e214e21e admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:01:50.787522 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a215bd-9315-4907-bdfa-83a4e214e21e admin admin] PATH_INFO: `/v3/users/2b86ea60f033481795115ea5b30bfcb6` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:01:50.800085 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a215bd-9315-4907-bdfa-83a4e214e21e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b86ea60f033481795115ea5b30bfcb6)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:01:50.801095 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: WARNING py.warnings [None req-b1a215bd-9315-4907-bdfa-83a4e214e21e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 20:01:50.801095 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: warnings.warn(msg) Oct 23 20:01:50.801095 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: Oct 23 20:01:50.802856 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a215bd-9315-4907-bdfa-83a4e214e21e admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:01:50.812410 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:01:50.870421 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-b1a215bd-9315-4907-bdfa-83a4e214e21e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b86ea60f033481795115ea5b30bfcb6'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:01:50.893412 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-b1a215bd-9315-4907-bdfa-83a4e214e21e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b86ea60f033481795115ea5b30bfcb6'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:01:50.903346 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-b1a215bd-9315-4907-bdfa-83a4e214e21e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b86ea60f033481795115ea5b30bfcb6'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:01:50.903773 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-b1a215bd-9315-4907-bdfa-83a4e214e21e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b86ea60f033481795115ea5b30bfcb6'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:01:50.917804 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-b1a215bd-9315-4907-bdfa-83a4e214e21e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b86ea60f033481795115ea5b30bfcb6'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:01:50.923664 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 592/1184] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:01:50 2022] DELETE /identity/v3/users/2b86ea60f033481795115ea5b30bfcb6 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:01:50.928733 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d315db-46e4-4090-995f-163943820e79 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:01:50.929188 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d315db-46e4-4090-995f-163943820e79 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:01:50.929296 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d315db-46e4-4090-995f-163943820e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:01:50.985825 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-13d315db-46e4-4090-995f-163943820e79 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:01:50.987045 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-13d315db-46e4-4090-995f-163943820e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:01:51.061406 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:01:51.074692 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 593/1185] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:01:51.084960 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a215bd-9315-4907-bdfa-83a4e214e21e admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:01:51.086219 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-b1a215bd-9315-4907-bdfa-83a4e214e21e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:01:51.185864 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a215bd-9315-4907-bdfa-83a4e214e21e admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:01:51.240099 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:01:51.260878 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af670b8-806d-4268-a27a-677188c535aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:01:51.261546 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af670b8-806d-4268-a27a-677188c535aa service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:01:51.261829 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af670b8-806d-4268-a27a-677188c535aa service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:01:51.262110 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af670b8-806d-4268-a27a-677188c535aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:01:51.263724 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7af670b8-806d-4268-a27a-677188c535aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:01:51.371206 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af670b8-806d-4268-a27a-677188c535aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:01:51.372634 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af670b8-806d-4268-a27a-677188c535aa service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:01:51.391346 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 593/1186] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:01:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:01:51.804407 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d315db-46e4-4090-995f-163943820e79 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:01:51.805688 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-13d315db-46e4-4090-995f-163943820e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:01:51.816950 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d315db-46e4-4090-995f-163943820e79 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:01:51.911825 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9ea26-d45f-4c83-91d0-9cea0048b21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:01:51.912453 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9ea26-d45f-4c83-91d0-9cea0048b21d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:01:51.912731 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9ea26-d45f-4c83-91d0-9cea0048b21d admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:01:51.912998 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9ea26-d45f-4c83-91d0-9cea0048b21d admin admin] PATH_INFO: `/v3/projects/16b8829fed62408bafc50ad32939bb25` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:01:51.922649 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb9ea26-d45f-4c83-91d0-9cea0048b21d admin admin] RBAC: Authorizing `identity:delete_project(project_id=16b8829fed62408bafc50ad32939bb25)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:01:51.923666 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: WARNING py.warnings [None req-0bb9ea26-d45f-4c83-91d0-9cea0048b21d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 20:01:51.923666 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: warnings.warn(msg) Oct 23 20:01:51.923666 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: Oct 23 20:01:51.924863 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb9ea26-d45f-4c83-91d0-9cea0048b21d admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:01:51.983264 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-0bb9ea26-d45f-4c83-91d0-9cea0048b21d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16b8829fed62408bafc50ad32939bb25', 'request_id': 'req-0bb9ea26-d45f-4c83-91d0-9cea0048b21d'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:01:51.993456 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-0bb9ea26-d45f-4c83-91d0-9cea0048b21d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16b8829fed62408bafc50ad32939bb25', 'request_id': 'req-0bb9ea26-d45f-4c83-91d0-9cea0048b21d'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:01:51.999529 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-0bb9ea26-d45f-4c83-91d0-9cea0048b21d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16b8829fed62408bafc50ad32939bb25', 'request_id': 'req-0bb9ea26-d45f-4c83-91d0-9cea0048b21d'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:01:52.029136 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-0bb9ea26-d45f-4c83-91d0-9cea0048b21d admin admin] The token cache is being invalidate because project 16b8829fed62408bafc50ad32939bb25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:01:52.029429 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-0bb9ea26-d45f-4c83-91d0-9cea0048b21d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:01:52.029955 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-0bb9ea26-d45f-4c83-91d0-9cea0048b21d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:01:52.031313 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 594/1187] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:01:51 2022] DELETE /identity/v3/projects/16b8829fed62408bafc50ad32939bb25 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:02:01.389030 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=62 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:02:02.027075 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:03:16.327296 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a12d4a-e0c1-4e6a-936f-9dca45ba8cc6 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:03:16.327863 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a12d4a-e0c1-4e6a-936f-9dca45ba8cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:03:16.327863 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a12d4a-e0c1-4e6a-936f-9dca45ba8cc6 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:03:16.329194 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 594/1188] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 20:03:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:03:16.334594 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02912e32-6114-42f7-8d2c-9316f1b02419 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:03:16.335038 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02912e32-6114-42f7-8d2c-9316f1b02419 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:03:16.335144 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02912e32-6114-42f7-8d2c-9316f1b02419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:03:16.415624 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-02912e32-6114-42f7-8d2c-9316f1b02419 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:03:16.416835 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-02912e32-6114-42f7-8d2c-9316f1b02419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:03:16.516565 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 595/1189] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 20:03:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:03:16.670443 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a12d4a-e0c1-4e6a-936f-9dca45ba8cc6 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:03:16.672639 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e2a12d4a-e0c1-4e6a-936f-9dca45ba8cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:03:16.791844 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a12d4a-e0c1-4e6a-936f-9dca45ba8cc6 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:03:16.797323 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b335a26-d34e-4415-aae7-3e16e4fb6149 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:03:16.797937 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b335a26-d34e-4415-aae7-3e16e4fb6149 service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:03:16.798217 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b335a26-d34e-4415-aae7-3e16e4fb6149 service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:03:16.798524 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b335a26-d34e-4415-aae7-3e16e4fb6149 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:03:16.806919 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b335a26-d34e-4415-aae7-3e16e4fb6149 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:03:16.808483 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b335a26-d34e-4415-aae7-3e16e4fb6149 service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:03:16.817680 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 595/1190] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:03:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:03:17.882806 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44a115-0698-4f47-aae6-a58e45f02605 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:03:17.883363 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44a115-0698-4f47-aae6-a58e45f02605 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:03:17.883363 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c44a115-0698-4f47-aae6-a58e45f02605 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:03:17.884704 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 596/1191] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 20:03:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:03:17.889339 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad0b58a-8f74-4a99-bb43-74700abcceff None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:03:17.889722 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad0b58a-8f74-4a99-bb43-74700abcceff None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:03:17.889879 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad0b58a-8f74-4a99-bb43-74700abcceff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:03:17.910748 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-9ad0b58a-8f74-4a99-bb43-74700abcceff None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:03:17.912017 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9ad0b58a-8f74-4a99-bb43-74700abcceff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:03:17.920364 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 596/1192] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 20:03:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:03:18.072308 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c44a115-0698-4f47-aae6-a58e45f02605 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:03:18.087005 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c44a115-0698-4f47-aae6-a58e45f02605 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:03:18.093428 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd0d95b-3768-4aa3-9908-9ad0238040d9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:03:18.094023 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0d95b-3768-4aa3-9908-9ad0238040d9 service nova] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:03:18.094329 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0d95b-3768-4aa3-9908-9ad0238040d9 service nova] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:03:18.094604 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd0d95b-3768-4aa3-9908-9ad0238040d9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:03:18.102493 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd0d95b-3768-4aa3-9908-9ad0238040d9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:03:18.104080 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd0d95b-3768-4aa3-9908-9ad0238040d9 service nova] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:03:18.113150 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 597/1193] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:03:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:03:18.433584 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e1331-7800-40e1-a889-35cef8e6af7c None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:03:18.433818 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e1331-7800-40e1-a889-35cef8e6af7c None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:03:18.434099 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828e1331-7800-40e1-a889-35cef8e6af7c None None] PATH_INFO: `/v3` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:03:18.435423 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 597/1194] 10.222.0.15 () {54 vars in 925 bytes} [Sun Oct 23 20:03:18 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:03:18.441479 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3ffc6-420d-4ff3-afb4-8d15bf80b2cd None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:03:18.441766 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3ffc6-420d-4ff3-afb4-8d15bf80b2cd None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:03:18.442037 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab3ffc6-420d-4ff3-afb4-8d15bf80b2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:03:18.501149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-9ab3ffc6-420d-4ff3-afb4-8d15bf80b2cd None None] MFA Rules not processed for user `312489c49a304ca9aacebd9843a4b65f`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:03:18.502371 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-9ab3ffc6-420d-4ff3-afb4-8d15bf80b2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:03:18.518976 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:03:18.575432 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 598/1195] 10.222.0.15 () {58 vars in 1015 bytes} [Sun Oct 23 20:03:18 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:03:22.134900 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dfb552-3d08-4515-ada8-1528e7b5c009 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:03:22.135596 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dfb552-3d08-4515-ada8-1528e7b5c009 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:03:22.135596 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dfb552-3d08-4515-ada8-1528e7b5c009 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:03:22.136839 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 598/1196] 10.222.0.13 () {54 vars in 961 bytes} [Sun Oct 23 20:03:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:03:22.141948 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cf31c-439b-4cf3-bfaf-489d61dcb746 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:03:22.142387 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cf31c-439b-4cf3-bfaf-489d61dcb746 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:03:22.142534 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6cf31c-439b-4cf3-bfaf-489d61dcb746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:03:22.161483 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-7a6cf31c-439b-4cf3-bfaf-489d61dcb746 None None] MFA Rules not processed for user `312489c49a304ca9aacebd9843a4b65f`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:03:22.162751 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-7a6cf31c-439b-4cf3-bfaf-489d61dcb746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:03:22.171170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 599/1197] 10.222.0.13 () {58 vars in 1058 bytes} [Sun Oct 23 20:03:22 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:03:22.179083 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dfb552-3d08-4515-ada8-1528e7b5c009 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:03:22.192310 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dfb552-3d08-4515-ada8-1528e7b5c009 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:03:22.198071 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25df4a11-5e0f-482a-9a1c-2c8448d97126 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312489c49a304ca9aacebd9843a4b65f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:03:22.198701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df4a11-5e0f-482a-9a1c-2c8448d97126 service placement] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:03:22.198983 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df4a11-5e0f-482a-9a1c-2c8448d97126 service placement] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:03:22.199258 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25df4a11-5e0f-482a-9a1c-2c8448d97126 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:03:22.207464 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25df4a11-5e0f-482a-9a1c-2c8448d97126 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:03:22.209071 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25df4a11-5e0f-482a-9a1c-2c8448d97126 service placement] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:03:22.218556 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 599/1198] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:03:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:03:29.541790 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ee526-3c04-47fa-97d3-beb3e3ea3caf None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:03:29.542390 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ee526-3c04-47fa-97d3-beb3e3ea3caf None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:03:29.542390 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3ee526-3c04-47fa-97d3-beb3e3ea3caf None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:03:29.543740 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 600/1199] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 20:03:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:03:29.548875 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cedf55-fe62-4b2c-bcea-a969c24100d0 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:03:29.549320 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cedf55-fe62-4b2c-bcea-a969c24100d0 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:03:29.549425 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cedf55-fe62-4b2c-bcea-a969c24100d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:03:29.569563 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-e8cedf55-fe62-4b2c-bcea-a969c24100d0 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:03:29.570770 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e8cedf55-fe62-4b2c-bcea-a969c24100d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:03:29.578366 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 600/1200] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 20:03:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:03:29.735433 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ee526-3c04-47fa-97d3-beb3e3ea3caf None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:03:29.751207 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3ee526-3c04-47fa-97d3-beb3e3ea3caf None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:03:29.756506 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17697c20-b5cb-41c1-951a-4d6878591b20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:03:29.757101 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17697c20-b5cb-41c1-951a-4d6878591b20 service nova] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:03:29.757377 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17697c20-b5cb-41c1-951a-4d6878591b20 service nova] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:03:29.757647 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17697c20-b5cb-41c1-951a-4d6878591b20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:03:29.766443 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17697c20-b5cb-41c1-951a-4d6878591b20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:03:29.768009 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17697c20-b5cb-41c1-951a-4d6878591b20 service nova] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:03:29.777122 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 601/1201] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:03:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:03:30.822549 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778ed7f-1013-4b86-8885-20adb6902f10 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:03:30.823133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778ed7f-1013-4b86-8885-20adb6902f10 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:03:30.823133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7778ed7f-1013-4b86-8885-20adb6902f10 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:03:30.824443 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 601/1202] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 20:03:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:03:30.829131 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572451a-e100-4176-ae85-4cc981ede409 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:03:30.829512 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572451a-e100-4176-ae85-4cc981ede409 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:03:30.829689 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b572451a-e100-4176-ae85-4cc981ede409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:03:30.849492 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-b572451a-e100-4176-ae85-4cc981ede409 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:03:30.850817 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-b572451a-e100-4176-ae85-4cc981ede409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:03:30.859245 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 602/1203] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 20:03:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:03:31.007024 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778ed7f-1013-4b86-8885-20adb6902f10 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:03:31.022037 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7778ed7f-1013-4b86-8885-20adb6902f10 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:03:31.027876 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fbf7bc-f146-43ae-8950-b82dddbf7a28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:03:31.028471 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fbf7bc-f146-43ae-8950-b82dddbf7a28 service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:03:31.028750 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fbf7bc-f146-43ae-8950-b82dddbf7a28 service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:03:31.029022 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fbf7bc-f146-43ae-8950-b82dddbf7a28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:03:31.037044 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fbf7bc-f146-43ae-8950-b82dddbf7a28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:03:31.038653 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fbf7bc-f146-43ae-8950-b82dddbf7a28 service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:03:31.047835 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 602/1204] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:03:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:03:34.935899 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b102311-8423-4bd5-8979-dd2817b59d3d None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:03:34.936366 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b102311-8423-4bd5-8979-dd2817b59d3d None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:03:34.936468 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b102311-8423-4bd5-8979-dd2817b59d3d None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:03:34.937805 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 603/1205] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 20:03:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:03:34.942981 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a726a-6ad0-44fc-8c54-a3b47dde4ca1 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:03:34.943661 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a726a-6ad0-44fc-8c54-a3b47dde4ca1 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:03:34.943786 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66a726a-6ad0-44fc-8c54-a3b47dde4ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:03:34.962736 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-c66a726a-6ad0-44fc-8c54-a3b47dde4ca1 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:03:34.963915 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c66a726a-6ad0-44fc-8c54-a3b47dde4ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:03:34.971529 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 603/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 20:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:03:35.261608 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b102311-8423-4bd5-8979-dd2817b59d3d None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:03:35.263392 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-3b102311-8423-4bd5-8979-dd2817b59d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:03:35.342364 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b102311-8423-4bd5-8979-dd2817b59d3d None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:03:35.345953 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b7f313-dfb8-4e37-8187-a4620afde0f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:03:35.346586 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b7f313-dfb8-4e37-8187-a4620afde0f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:03:35.346865 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b7f313-dfb8-4e37-8187-a4620afde0f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:03:35.347135 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b7f313-dfb8-4e37-8187-a4620afde0f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:03:35.355981 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b7f313-dfb8-4e37-8187-a4620afde0f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:03:35.357394 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b7f313-dfb8-4e37-8187-a4620afde0f3 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:03:35.366671 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 604/1207] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:03:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:03:37.503266 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663191f0-75d5-47a9-846b-c4ce9d9cdd2e None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:03:37.503823 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663191f0-75d5-47a9-846b-c4ce9d9cdd2e None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:03:37.503823 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663191f0-75d5-47a9-846b-c4ce9d9cdd2e None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:03:37.505235 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 604/1208] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 20:03:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:03:37.510508 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8247b7-b4de-4afd-9eaa-b485824344c5 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:03:37.510962 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8247b7-b4de-4afd-9eaa-b485824344c5 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:03:37.511099 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8247b7-b4de-4afd-9eaa-b485824344c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:03:37.530939 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-3f8247b7-b4de-4afd-9eaa-b485824344c5 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:03:37.532137 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-3f8247b7-b4de-4afd-9eaa-b485824344c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:03:37.540612 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 605/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 20:03:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:03:37.831064 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663191f0-75d5-47a9-846b-c4ce9d9cdd2e None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:03:37.842438 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663191f0-75d5-47a9-846b-c4ce9d9cdd2e None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:03:37.846120 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202d99d-6519-4655-8853-02a7eef4aff5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:03:37.846754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202d99d-6519-4655-8853-02a7eef4aff5 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:03:37.847036 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202d99d-6519-4655-8853-02a7eef4aff5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:03:37.847338 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202d99d-6519-4655-8853-02a7eef4aff5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:03:37.856234 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2202d99d-6519-4655-8853-02a7eef4aff5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:03:37.857680 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2202d99d-6519-4655-8853-02a7eef4aff5 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:03:37.866949 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 605/1210] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:03:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:03:41.510733 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c89013-ada7-485e-ba2a-5abd8c3f1f32 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:03:41.511305 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c89013-ada7-485e-ba2a-5abd8c3f1f32 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:03:41.511305 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c89013-ada7-485e-ba2a-5abd8c3f1f32 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:03:41.512636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 606/1211] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 20:03:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:03:41.517908 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff36bf5-1a62-4aec-90ea-8c73bd5ef919 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:03:41.518384 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff36bf5-1a62-4aec-90ea-8c73bd5ef919 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:03:41.518502 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff36bf5-1a62-4aec-90ea-8c73bd5ef919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:03:41.526739 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=81 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:03:41.538890 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-7ff36bf5-1a62-4aec-90ea-8c73bd5ef919 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:03:41.540120 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7ff36bf5-1a62-4aec-90ea-8c73bd5ef919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:03:41.547896 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 606/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 20:03:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:03:41.850569 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c89013-ada7-485e-ba2a-5abd8c3f1f32 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:03:41.852285 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-81c89013-ada7-485e-ba2a-5abd8c3f1f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:03:41.915667 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:03:41.929175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c89013-ada7-485e-ba2a-5abd8c3f1f32 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:03:41.933242 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fd6a8-b046-494c-8a6f-130e5b07e153 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:03:41.933846 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fd6a8-b046-494c-8a6f-130e5b07e153 service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:03:41.934127 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fd6a8-b046-494c-8a6f-130e5b07e153 service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:03:41.934444 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fd6a8-b046-494c-8a6f-130e5b07e153 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:03:41.943209 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2fd6a8-b046-494c-8a6f-130e5b07e153 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:03:41.944884 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2fd6a8-b046-494c-8a6f-130e5b07e153 service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:03:41.954671 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 607/1213] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:03:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:03:44.084733 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0ceb6-61b4-40d7-b096-a927920f6c8e None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:03:44.085275 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0ceb6-61b4-40d7-b096-a927920f6c8e None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:03:44.085275 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f0ceb6-61b4-40d7-b096-a927920f6c8e None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:03:44.086699 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 607/1214] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 20:03:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:03:44.092571 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecf2cf-07f0-43a8-a99e-ebcc4f24b4c5 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:03:44.092988 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecf2cf-07f0-43a8-a99e-ebcc4f24b4c5 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:03:44.093124 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecf2cf-07f0-43a8-a99e-ebcc4f24b4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:03:44.112974 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-acecf2cf-07f0-43a8-a99e-ebcc4f24b4c5 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:03:44.114177 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-acecf2cf-07f0-43a8-a99e-ebcc4f24b4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:03:44.122916 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 608/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 20:03:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:03:44.425700 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0ceb6-61b4-40d7-b096-a927920f6c8e None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:03:44.427420 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e3f0ceb6-61b4-40d7-b096-a927920f6c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:03:44.440312 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f0ceb6-61b4-40d7-b096-a927920f6c8e None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:03:44.444109 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e264ca88-34d7-4206-a395-e39f611e3588 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:03:44.444708 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264ca88-34d7-4206-a395-e39f611e3588 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:03:44.444986 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264ca88-34d7-4206-a395-e39f611e3588 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:03:44.445261 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e264ca88-34d7-4206-a395-e39f611e3588 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:03:44.454362 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e264ca88-34d7-4206-a395-e39f611e3588 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:03:44.455788 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e264ca88-34d7-4206-a395-e39f611e3588 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:03:44.465628 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 608/1216] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:03:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:03:54.108114 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=11 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:03:54.460906 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=8 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:06.189555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413cd9a8-1764-4909-93fe-b6685e5952b0 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:06.190198 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413cd9a8-1764-4909-93fe-b6685e5952b0 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:06.190198 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413cd9a8-1764-4909-93fe-b6685e5952b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:06.202974 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083bf002-638d-41a4-8901-8c0c3c8c22a6 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:06.204409 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083bf002-638d-41a4-8901-8c0c3c8c22a6 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:06.204409 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083bf002-638d-41a4-8901-8c0c3c8c22a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:06.219328 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-413cd9a8-1764-4909-93fe-b6685e5952b0 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:06.220572 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-413cd9a8-1764-4909-93fe-b6685e5952b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:06.232011 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 609/1217] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:06.239339 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-083bf002-638d-41a4-8901-8c0c3c8c22a6 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:06.240605 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-083bf002-638d-41a4-8901-8c0c3c8c22a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:06.244478 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193a88a-3460-401f-994f-20ad23be278c None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:06.244781 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193a88a-3460-401f-994f-20ad23be278c None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:06.245029 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193a88a-3460-401f-994f-20ad23be278c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:06.255134 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 609/1218] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:05:06.268015 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a571ee3-79d3-447b-a635-f1331cbcec8d None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:06.268357 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a571ee3-79d3-447b-a635-f1331cbcec8d None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:06.268586 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a571ee3-79d3-447b-a635-f1331cbcec8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:06.278860 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-8193a88a-3460-401f-994f-20ad23be278c None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:06.279987 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-8193a88a-3460-401f-994f-20ad23be278c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:06.298491 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 610/1219] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:06.304063 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193a88a-3460-401f-994f-20ad23be278c None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:06.305433 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-0a571ee3-79d3-447b-a635-f1331cbcec8d None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:06.306695 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-0a571ee3-79d3-447b-a635-f1331cbcec8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:06.319973 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 610/1220] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:05:06.321358 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae0669-973a-47cb-bb12-8cf68ed3c78d None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:06.321646 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae0669-973a-47cb-bb12-8cf68ed3c78d None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:06.321924 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ae0669-973a-47cb-bb12-8cf68ed3c78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:06.325098 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193a88a-3460-401f-994f-20ad23be278c None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:06.330042 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b25a1-62c0-4fc3-a2aa-5ccf8e7c1a11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:06.330697 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b25a1-62c0-4fc3-a2aa-5ccf8e7c1a11 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:06.330978 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b25a1-62c0-4fc3-a2aa-5ccf8e7c1a11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:06.331249 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b25a1-62c0-4fc3-a2aa-5ccf8e7c1a11 admin admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:06.332099 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b25a1-62c0-4fc3-a2aa-5ccf8e7c1a11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:05:06.332490 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b25a1-62c0-4fc3-a2aa-5ccf8e7c1a11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:06.334326 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b25a1-62c0-4fc3-a2aa-5ccf8e7c1a11 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:06.360814 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 611/1221] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:05:06.361741 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b25a1-62c0-4fc3-a2aa-5ccf8e7c1a11 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:06.363731 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-04ae0669-973a-47cb-bb12-8cf68ed3c78d None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:06.364992 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-04ae0669-973a-47cb-bb12-8cf68ed3c78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:06.373881 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b25a1-62c0-4fc3-a2aa-5ccf8e7c1a11 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:06.379678 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7eb600-ce4b-40ad-a439-4abee59e47ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:06.380318 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7eb600-ce4b-40ad-a439-4abee59e47ad admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:06.380602 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7eb600-ce4b-40ad-a439-4abee59e47ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:06.380878 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7eb600-ce4b-40ad-a439-4abee59e47ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:06.381549 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7eb600-ce4b-40ad-a439-4abee59e47ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:05:06.381853 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7eb600-ce4b-40ad-a439-4abee59e47ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:06.383580 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7eb600-ce4b-40ad-a439-4abee59e47ad admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:06.385710 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 611/1222] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:06.386673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae0669-973a-47cb-bb12-8cf68ed3c78d None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:06.405638 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ae0669-973a-47cb-bb12-8cf68ed3c78d None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:06.414836 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 612/1223] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:05:06.416022 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73aa2e-8d43-4675-855b-9e3dc83c26c4 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:06.416224 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5bba00-c635-478a-8acd-1f544c299fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:06.416364 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73aa2e-8d43-4675-855b-9e3dc83c26c4 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:06.416557 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73aa2e-8d43-4675-855b-9e3dc83c26c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:06.416806 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5bba00-c635-478a-8acd-1f544c299fcd admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:06.417094 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5bba00-c635-478a-8acd-1f544c299fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:06.417373 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5bba00-c635-478a-8acd-1f544c299fcd admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:06.418019 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5bba00-c635-478a-8acd-1f544c299fcd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:06.419957 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5bba00-c635-478a-8acd-1f544c299fcd admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:06.448785 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-af73aa2e-8d43-4675-855b-9e3dc83c26c4 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:06.449997 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-af73aa2e-8d43-4675-855b-9e3dc83c26c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:06.466544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 613/1224] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:06.467371 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af73aa2e-8d43-4675-855b-9e3dc83c26c4 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:06.485196 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af73aa2e-8d43-4675-855b-9e3dc83c26c4 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:06.490263 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737ee25-910d-4a2c-bc3b-f7dc7e5b9888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:06.495129 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737ee25-910d-4a2c-bc3b-f7dc7e5b9888 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:06.495505 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737ee25-910d-4a2c-bc3b-f7dc7e5b9888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:06.495784 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8737ee25-910d-4a2c-bc3b-f7dc7e5b9888 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:06.496428 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8737ee25-910d-4a2c-bc3b-f7dc7e5b9888 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:06.497442 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: WARNING py.warnings [None req-8737ee25-910d-4a2c-bc3b-f7dc7e5b9888 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 20:05:06.497442 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: warnings.warn(msg) Oct 23 20:05:06.497442 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: Oct 23 20:05:06.498811 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8737ee25-910d-4a2c-bc3b-f7dc7e5b9888 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:06.500132 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 612/1225] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:05:06 2022] POST /identity/v3/projects => generated 379 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:06.501058 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5bba00-c635-478a-8acd-1f544c299fcd admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:06.546781 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 614/1226] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:05:06 2022] POST /identity/v3/projects => generated 389 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:06.547565 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737ee25-910d-4a2c-bc3b-f7dc7e5b9888 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:06.590655 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5bba00-c635-478a-8acd-1f544c299fcd admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:06.639478 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8737ee25-910d-4a2c-bc3b-f7dc7e5b9888 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:06.644761 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dec0fdf-c160-4917-954a-e7f3c4f6e834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:06.645382 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec0fdf-c160-4917-954a-e7f3c4f6e834 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:06.645644 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec0fdf-c160-4917-954a-e7f3c4f6e834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:06.645959 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dec0fdf-c160-4917-954a-e7f3c4f6e834 admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:06.646896 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dec0fdf-c160-4917-954a-e7f3c4f6e834 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:06.649015 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dec0fdf-c160-4917-954a-e7f3c4f6e834 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:06.655267 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d44229-abb7-433f-b7dd-1ec679afbaea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:06.655900 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d44229-abb7-433f-b7dd-1ec679afbaea admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:06.656185 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d44229-abb7-433f-b7dd-1ec679afbaea admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:06.656595 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d44229-abb7-433f-b7dd-1ec679afbaea admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:06.657251 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d44229-abb7-433f-b7dd-1ec679afbaea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:05:06.657614 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d44229-abb7-433f-b7dd-1ec679afbaea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:06.658653 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: WARNING py.warnings [None req-84d44229-abb7-433f-b7dd-1ec679afbaea admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 23 20:05:06.658653 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: warnings.warn(msg) Oct 23 20:05:06.658653 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: Oct 23 20:05:06.659539 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d44229-abb7-433f-b7dd-1ec679afbaea admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:06.676444 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 613/1227] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:05:06.678215 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce8822-3c1f-4f2f-8626-4e35d130a94b None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:06.678432 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 615/1228] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:05:06 2022] POST /identity/v3/users => generated 290 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:06.678965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce8822-3c1f-4f2f-8626-4e35d130a94b None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:06.679536 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce8822-3c1f-4f2f-8626-4e35d130a94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:06.680546 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667517bf-82bd-410b-bc38-a9a2c809acbc None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:06.680836 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667517bf-82bd-410b-bc38-a9a2c809acbc None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:06.681106 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667517bf-82bd-410b-bc38-a9a2c809acbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:06.701777 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-5bce8822-3c1f-4f2f-8626-4e35d130a94b None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:06.702402 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-667517bf-82bd-410b-bc38-a9a2c809acbc None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:06.703035 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5bce8822-3c1f-4f2f-8626-4e35d130a94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:06.703577 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-667517bf-82bd-410b-bc38-a9a2c809acbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:06.711057 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 614/1229] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:06.711546 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 616/1230] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:06.711814 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce8822-3c1f-4f2f-8626-4e35d130a94b None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:06.720208 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697310c-3482-4fc8-9167-c12a95b2076e None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:06.720728 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697310c-3482-4fc8-9167-c12a95b2076e None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:06.721238 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4697310c-3482-4fc8-9167-c12a95b2076e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:06.722012 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce8822-3c1f-4f2f-8626-4e35d130a94b None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:06.726839 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baaa52e-59b5-48c8-9910-5800ed500559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:06.727444 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baaa52e-59b5-48c8-9910-5800ed500559 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:06.727723 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baaa52e-59b5-48c8-9910-5800ed500559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:06.727996 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baaa52e-59b5-48c8-9910-5800ed500559 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:06.728633 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baaa52e-59b5-48c8-9910-5800ed500559 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:06.729671 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: WARNING py.warnings [None req-0baaa52e-59b5-48c8-9910-5800ed500559 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 20:05:06.729671 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: warnings.warn(msg) Oct 23 20:05:06.729671 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: Oct 23 20:05:06.730921 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baaa52e-59b5-48c8-9910-5800ed500559 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:06.741107 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-4697310c-3482-4fc8-9167-c12a95b2076e None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:06.743260 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-4697310c-3482-4fc8-9167-c12a95b2076e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:06.752663 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 617/1231] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:06.757234 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ae99f-db30-4e6e-abaa-e6167eeade07 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:06.757373 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 615/1232] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:05:06 2022] POST /identity/v3/users => generated 295 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:06.757498 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ae99f-db30-4e6e-abaa-e6167eeade07 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:06.757498 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554ae99f-db30-4e6e-abaa-e6167eeade07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:06.757684 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459781a-f0c7-4d09-8ff4-2a8a58a0ac6a None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:06.757684 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459781a-f0c7-4d09-8ff4-2a8a58a0ac6a None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:06.757684 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c459781a-f0c7-4d09-8ff4-2a8a58a0ac6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:06.775102 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-554ae99f-db30-4e6e-abaa-e6167eeade07 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:06.776310 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-554ae99f-db30-4e6e-abaa-e6167eeade07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:06.776446 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-c459781a-f0c7-4d09-8ff4-2a8a58a0ac6a None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:06.777566 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c459781a-f0c7-4d09-8ff4-2a8a58a0ac6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:06.784273 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 618/1233] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:06.785271 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 616/1234] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:06.786178 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554ae99f-db30-4e6e-abaa-e6167eeade07 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:06.791169 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c459781a-f0c7-4d09-8ff4-2a8a58a0ac6a None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:06.799686 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554ae99f-db30-4e6e-abaa-e6167eeade07 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:06.801053 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c459781a-f0c7-4d09-8ff4-2a8a58a0ac6a None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:06.804427 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8030115b-3ae3-465d-9b6c-52a7966f6ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:06.805568 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030115b-3ae3-465d-9b6c-52a7966f6ea4 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:06.805849 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030115b-3ae3-465d-9b6c-52a7966f6ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:06.806116 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030115b-3ae3-465d-9b6c-52a7966f6ea4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:06.806356 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8296b5d-387f-4723-9c7d-574924747f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:06.806793 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8030115b-3ae3-465d-9b6c-52a7966f6ea4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:06.806959 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8296b5d-387f-4723-9c7d-574924747f57 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:06.807239 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8296b5d-387f-4723-9c7d-574924747f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:06.807513 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8296b5d-387f-4723-9c7d-574924747f57 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:06.808147 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8296b5d-387f-4723-9c7d-574924747f57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:06.808386 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8030115b-3ae3-465d-9b6c-52a7966f6ea4 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:06.810030 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8296b5d-387f-4723-9c7d-574924747f57 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:06.819194 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 619/1235] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:06.821431 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e57cd6-63af-4fd0-92bf-459df1c791b8 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:06.821722 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e57cd6-63af-4fd0-92bf-459df1c791b8 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:06.822577 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e57cd6-63af-4fd0-92bf-459df1c791b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:06.831749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:06.840956 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 617/1236] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:05:06 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:06.842366 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb6267-b822-46ca-8562-2c98411b5f12 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:06.842672 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb6267-b822-46ca-8562-2c98411b5f12 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:06.842945 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cb6267-b822-46ca-8562-2c98411b5f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:06.845120 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-d7e57cd6-63af-4fd0-92bf-459df1c791b8 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:06.846466 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-d7e57cd6-63af-4fd0-92bf-459df1c791b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:06.861752 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-64cb6267-b822-46ca-8562-2c98411b5f12 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:06.862995 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-64cb6267-b822-46ca-8562-2c98411b5f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:06.892977 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:06.908300 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 620/1237] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:06.909189 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e57cd6-63af-4fd0-92bf-459df1c791b8 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:06.918420 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e57cd6-63af-4fd0-92bf-459df1c791b8 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:06.922938 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39012208-09f8-4a1d-939c-ab6dd234c67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:06.923622 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39012208-09f8-4a1d-939c-ab6dd234c67e admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:06.924393 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 618/1238] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:06.924511 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39012208-09f8-4a1d-939c-ab6dd234c67e admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:06.924612 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39012208-09f8-4a1d-939c-ab6dd234c67e admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:06.925412 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85e066-dbc1-4b68-ab9b-1ce3cf85396d None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:06.925569 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39012208-09f8-4a1d-939c-ab6dd234c67e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:06.925703 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85e066-dbc1-4b68-ab9b-1ce3cf85396d None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:06.925978 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85e066-dbc1-4b68-ab9b-1ce3cf85396d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:06.927207 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39012208-09f8-4a1d-939c-ab6dd234c67e admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:06.941091 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 621/1239] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:06.941091 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41230f1d-1599-44d9-aaa3-681b6178bbf2 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:06.941091 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41230f1d-1599-44d9-aaa3-681b6178bbf2 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:06.941091 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41230f1d-1599-44d9-aaa3-681b6178bbf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:06.948682 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-5d85e066-dbc1-4b68-ab9b-1ce3cf85396d None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:06.950752 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5d85e066-dbc1-4b68-ab9b-1ce3cf85396d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:06.958510 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 619/1240] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:06.959297 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-41230f1d-1599-44d9-aaa3-681b6178bbf2 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:06.960564 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-41230f1d-1599-44d9-aaa3-681b6178bbf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:06.961194 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e90f6-4abc-46c3-a8cd-0cc858020fbc None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:06.961478 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e90f6-4abc-46c3-a8cd-0cc858020fbc None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:06.961771 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83e90f6-4abc-46c3-a8cd-0cc858020fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:06.968815 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 622/1241] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:06.969588 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41230f1d-1599-44d9-aaa3-681b6178bbf2 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:06.979372 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41230f1d-1599-44d9-aaa3-681b6178bbf2 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:06.982448 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-b83e90f6-4abc-46c3-a8cd-0cc858020fbc None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:06.983622 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-b83e90f6-4abc-46c3-a8cd-0cc858020fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:06.983932 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647f1e9-5317-492f-b0eb-048a6df90f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:06.984517 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647f1e9-5317-492f-b0eb-048a6df90f1d admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:06.984794 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647f1e9-5317-492f-b0eb-048a6df90f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:06.985061 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8647f1e9-5317-492f-b0eb-048a6df90f1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:06.985754 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8647f1e9-5317-492f-b0eb-048a6df90f1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:06.987344 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8647f1e9-5317-492f-b0eb-048a6df90f1d admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:06.994582 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 620/1242] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:06.995563 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e90f6-4abc-46c3-a8cd-0cc858020fbc None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:07.000918 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 623/1243] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:07.000918 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647f1e9-5317-492f-b0eb-048a6df90f1d admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:07.005379 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83e90f6-4abc-46c3-a8cd-0cc858020fbc None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:07.010265 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8647f1e9-5317-492f-b0eb-048a6df90f1d admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:07.013516 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af9044-e89c-419e-a4ac-f151777a2590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:07.014123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af9044-e89c-419e-a4ac-f151777a2590 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:07.014432 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af9044-e89c-419e-a4ac-f151777a2590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:07.014703 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13af9044-e89c-419e-a4ac-f151777a2590 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:07.015337 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13af9044-e89c-419e-a4ac-f151777a2590 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:07.015611 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388f7cf-ec63-4e41-9854-27b94d02ae5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:07.016205 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388f7cf-ec63-4e41-9854-27b94d02ae5f admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:07.016481 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388f7cf-ec63-4e41-9854-27b94d02ae5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:07.016755 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388f7cf-ec63-4e41-9854-27b94d02ae5f admin admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:07.017303 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13af9044-e89c-419e-a4ac-f151777a2590 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:07.017397 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388f7cf-ec63-4e41-9854-27b94d02ae5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:05:07.017674 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388f7cf-ec63-4e41-9854-27b94d02ae5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:07.019272 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388f7cf-ec63-4e41-9854-27b94d02ae5f admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:07.032455 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 624/1244] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:05:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:05:07.033459 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388f7cf-ec63-4e41-9854-27b94d02ae5f admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:07.043325 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 621/1245] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:05:06 2022] POST /identity/v3/users => generated 286 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:07.043985 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388f7cf-ec63-4e41-9854-27b94d02ae5f admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:07.044423 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af9044-e89c-419e-a4ac-f151777a2590 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:07.048759 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280fe6a9-b7b7-4d92-a7d9-1cfaf717fbe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:07.049356 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280fe6a9-b7b7-4d92-a7d9-1cfaf717fbe3 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:07.049636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280fe6a9-b7b7-4d92-a7d9-1cfaf717fbe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:07.049959 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280fe6a9-b7b7-4d92-a7d9-1cfaf717fbe3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:07.050603 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280fe6a9-b7b7-4d92-a7d9-1cfaf717fbe3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:05:07.050909 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280fe6a9-b7b7-4d92-a7d9-1cfaf717fbe3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:07.052493 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280fe6a9-b7b7-4d92-a7d9-1cfaf717fbe3 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:07.053870 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13af9044-e89c-419e-a4ac-f151777a2590 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:07.058521 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9511fd7-0935-4523-bf68-9683f23763c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:07.059093 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9511fd7-0935-4523-bf68-9683f23763c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:07.059368 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9511fd7-0935-4523-bf68-9683f23763c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:07.059640 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9511fd7-0935-4523-bf68-9683f23763c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:07.060268 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9511fd7-0935-4523-bf68-9683f23763c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:07.061857 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9511fd7-0935-4523-bf68-9683f23763c1 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:07.063939 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 625/1246] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:05:07.064795 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280fe6a9-b7b7-4d92-a7d9-1cfaf717fbe3 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:07.074966 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280fe6a9-b7b7-4d92-a7d9-1cfaf717fbe3 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:07.082525 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 622/1247] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:07.082678 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5034810-1ad1-452b-8009-cf12c341c5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:07.082678 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5034810-1ad1-452b-8009-cf12c341c5f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:07.082878 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9511fd7-0935-4523-bf68-9683f23763c1 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:07.082964 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5034810-1ad1-452b-8009-cf12c341c5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:07.082964 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5034810-1ad1-452b-8009-cf12c341c5f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:07.082964 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5034810-1ad1-452b-8009-cf12c341c5f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:05:07.082964 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5034810-1ad1-452b-8009-cf12c341c5f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:07.083350 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5034810-1ad1-452b-8009-cf12c341c5f7 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:07.090406 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9511fd7-0935-4523-bf68-9683f23763c1 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:07.094953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ad6b0-1b54-4dfd-b6fd-2667c0e8f41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:07.095528 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ad6b0-1b54-4dfd-b6fd-2667c0e8f41b admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:07.102974 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 626/1248] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:05:07.103128 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ad6b0-1b54-4dfd-b6fd-2667c0e8f41b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:07.103128 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66ad6b0-1b54-4dfd-b6fd-2667c0e8f41b admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:07.103308 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5034810-1ad1-452b-8009-cf12c341c5f7 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:07.103397 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66ad6b0-1b54-4dfd-b6fd-2667c0e8f41b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:05:07.103397 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66ad6b0-1b54-4dfd-b6fd-2667c0e8f41b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:07.103397 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66ad6b0-1b54-4dfd-b6fd-2667c0e8f41b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:07.105927 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5034810-1ad1-452b-8009-cf12c341c5f7 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:07.110497 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c667893e-a9ac-42f3-ba52-b31d78a57da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:07.111075 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667893e-a9ac-42f3-ba52-b31d78a57da2 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:07.111358 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667893e-a9ac-42f3-ba52-b31d78a57da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:07.111635 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c667893e-a9ac-42f3-ba52-b31d78a57da2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:07.112268 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c667893e-a9ac-42f3-ba52-b31d78a57da2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:05:07.112568 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c667893e-a9ac-42f3-ba52-b31d78a57da2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:07.113641 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 623/1249] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:05:07.114287 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ad6b0-1b54-4dfd-b6fd-2667c0e8f41b admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:07.114442 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c667893e-a9ac-42f3-ba52-b31d78a57da2 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:07.126008 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 627/1250] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:05:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:05:07.126814 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c667893e-a9ac-42f3-ba52-b31d78a57da2 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:07.129549 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66ad6b0-1b54-4dfd-b6fd-2667c0e8f41b admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:07.134927 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19622aaa-fbe3-49ff-8557-03dc390639cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:07.135596 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19622aaa-fbe3-49ff-8557-03dc390639cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:07.135884 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19622aaa-fbe3-49ff-8557-03dc390639cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:07.136150 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19622aaa-fbe3-49ff-8557-03dc390639cb admin admin] PATH_INFO: `/v3/projects/dd03c05196b64299bf41b7d4c412070a/users/a571b802668942e2b612ef125fe34ea6/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:07.138432 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c667893e-a9ac-42f3-ba52-b31d78a57da2 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:07.143954 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fcee67-3237-4be0-8359-fe5c3145dbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:07.144993 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fcee67-3237-4be0-8359-fe5c3145dbc3 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:07.145282 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fcee67-3237-4be0-8359-fe5c3145dbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:07.145557 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fcee67-3237-4be0-8359-fe5c3145dbc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:07.146185 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fcee67-3237-4be0-8359-fe5c3145dbc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:07.148113 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fcee67-3237-4be0-8359-fe5c3145dbc3 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:07.150758 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19622aaa-fbe3-49ff-8557-03dc390639cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd03c05196b64299bf41b7d4c412070a, user_id=a571b802668942e2b612ef125fe34ea6, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:07.153450 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19622aaa-fbe3-49ff-8557-03dc390639cb admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:07.167322 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 624/1251] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:07 2022] PUT /identity/v3/projects/dd03c05196b64299bf41b7d4c412070a/users/a571b802668942e2b612ef125fe34ea6/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:07.168264 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19622aaa-fbe3-49ff-8557-03dc390639cb admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:07.179167 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 628/1252] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:05:07 2022] POST /identity/v3/projects => generated 393 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:07.179167 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fcee67-3237-4be0-8359-fe5c3145dbc3 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:07.233760 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19622aaa-fbe3-49ff-8557-03dc390639cb admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:07.238422 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3207ff05-0642-43f0-b2d2-cdc23c8883d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:07.239029 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3207ff05-0642-43f0-b2d2-cdc23c8883d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:07.239310 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3207ff05-0642-43f0-b2d2-cdc23c8883d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:07.239586 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3207ff05-0642-43f0-b2d2-cdc23c8883d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:07.240067 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fcee67-3237-4be0-8359-fe5c3145dbc3 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:07.240469 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3207ff05-0642-43f0-b2d2-cdc23c8883d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:07.241803 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3207ff05-0642-43f0-b2d2-cdc23c8883d3 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:07.245194 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59595261-fbff-4b0b-9559-b3a5747cdec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:07.245803 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59595261-fbff-4b0b-9559-b3a5747cdec8 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:07.246089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59595261-fbff-4b0b-9559-b3a5747cdec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:07.246408 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59595261-fbff-4b0b-9559-b3a5747cdec8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:07.247036 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59595261-fbff-4b0b-9559-b3a5747cdec8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:07.248954 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59595261-fbff-4b0b-9559-b3a5747cdec8 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:07.251840 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 625/1253] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:07.254880 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3207ff05-0642-43f0-b2d2-cdc23c8883d3 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:07.262678 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3207ff05-0642-43f0-b2d2-cdc23c8883d3 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:07.267928 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f72e5-622d-44f0-88ff-2df2fb43b825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:07.268635 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f72e5-622d-44f0-88ff-2df2fb43b825 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:07.268912 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f72e5-622d-44f0-88ff-2df2fb43b825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:07.269223 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f72e5-622d-44f0-88ff-2df2fb43b825 admin admin] PATH_INFO: `/v3/projects/4023fb7431ad496fa044b19190f014d8/users/7cef68ee160348dea427e6c46b775e2d/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:07.279910 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 629/1254] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:05:07 2022] POST /identity/v3/projects => generated 375 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:07.279910 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59595261-fbff-4b0b-9559-b3a5747cdec8 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:07.284072 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:07.287798 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4f72e5-622d-44f0-88ff-2df2fb43b825 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4023fb7431ad496fa044b19190f014d8, user_id=7cef68ee160348dea427e6c46b775e2d, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:07.291654 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4f72e5-622d-44f0-88ff-2df2fb43b825 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:07.305114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 626/1255] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:07 2022] PUT /identity/v3/projects/4023fb7431ad496fa044b19190f014d8/users/7cef68ee160348dea427e6c46b775e2d/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:07.309379 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f72e5-622d-44f0-88ff-2df2fb43b825 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:07.314375 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:07.346168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59595261-fbff-4b0b-9559-b3a5747cdec8 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:07.370376 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f72e5-622d-44f0-88ff-2df2fb43b825 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:07.375395 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463ca80-0a7a-48ee-b9c4-88c6e78effb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:07.376005 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463ca80-0a7a-48ee-b9c4-88c6e78effb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:07.376286 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463ca80-0a7a-48ee-b9c4-88c6e78effb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:07.376558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8463ca80-0a7a-48ee-b9c4-88c6e78effb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:07.377184 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463ca80-0a7a-48ee-b9c4-88c6e78effb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:07.379122 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8463ca80-0a7a-48ee-b9c4-88c6e78effb8 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:07.402545 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea8d565-22ff-4bb7-9dbc-d68e31aeb48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:07.403161 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8d565-22ff-4bb7-9dbc-d68e31aeb48b admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:07.403442 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8d565-22ff-4bb7-9dbc-d68e31aeb48b admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:07.403715 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea8d565-22ff-4bb7-9dbc-d68e31aeb48b admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:07.404361 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea8d565-22ff-4bb7-9dbc-d68e31aeb48b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:07.409411 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 627/1256] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:05:07 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:07.409557 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea8d565-22ff-4bb7-9dbc-d68e31aeb48b admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:07.409653 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463ca80-0a7a-48ee-b9c4-88c6e78effb8 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:07.433823 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 630/1257] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:05:07 2022] POST /identity/v3/projects => generated 383 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:07.435167 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe56fd44-8561-4cb8-afdd-6870cd3ed8e8 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:07.435457 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe56fd44-8561-4cb8-afdd-6870cd3ed8e8 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:07.435737 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe56fd44-8561-4cb8-afdd-6870cd3ed8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:07.454083 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-fe56fd44-8561-4cb8-afdd-6870cd3ed8e8 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:07.455314 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-fe56fd44-8561-4cb8-afdd-6870cd3ed8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:07.473326 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8463ca80-0a7a-48ee-b9c4-88c6e78effb8 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:07.515463 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 631/1258] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:07.516468 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe56fd44-8561-4cb8-afdd-6870cd3ed8e8 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:07.526762 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe56fd44-8561-4cb8-afdd-6870cd3ed8e8 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:07.531340 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba152723-b36a-4db5-8304-fb5ee7aaa5cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:07.531938 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba152723-b36a-4db5-8304-fb5ee7aaa5cb admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:07.532216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba152723-b36a-4db5-8304-fb5ee7aaa5cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:07.532485 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba152723-b36a-4db5-8304-fb5ee7aaa5cb admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:07.533116 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba152723-b36a-4db5-8304-fb5ee7aaa5cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:07.535055 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba152723-b36a-4db5-8304-fb5ee7aaa5cb admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:07.535316 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a43a0-58f6-4dd1-9e40-f9e317f99adc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:07.535934 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a43a0-58f6-4dd1-9e40-f9e317f99adc admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:07.536217 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a43a0-58f6-4dd1-9e40-f9e317f99adc admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:07.536494 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a43a0-58f6-4dd1-9e40-f9e317f99adc admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:07.537124 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17a43a0-58f6-4dd1-9e40-f9e317f99adc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:07.539629 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17a43a0-58f6-4dd1-9e40-f9e317f99adc admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:07.557766 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 632/1259] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:05:07 2022] POST /identity/v3/users => generated 297 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:07.558765 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba152723-b36a-4db5-8304-fb5ee7aaa5cb admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:07.569932 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 628/1260] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:05:07 2022] POST /identity/v3/projects => generated 401 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:07.569932 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a43a0-58f6-4dd1-9e40-f9e317f99adc admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:07.570204 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba152723-b36a-4db5-8304-fb5ee7aaa5cb admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:07.578799 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:07.613204 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:07.626357 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51abdac5-17c6-456c-b0b5-93120e9c76f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:07.626966 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abdac5-17c6-456c-b0b5-93120e9c76f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:07.627291 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abdac5-17c6-456c-b0b5-93120e9c76f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:07.627565 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51abdac5-17c6-456c-b0b5-93120e9c76f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:07.628239 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51abdac5-17c6-456c-b0b5-93120e9c76f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:07.629896 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51abdac5-17c6-456c-b0b5-93120e9c76f7 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:07.633790 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a43a0-58f6-4dd1-9e40-f9e317f99adc admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:07.638518 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af793503-0626-42cf-a100-86e7b2a5bce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:07.639125 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af793503-0626-42cf-a100-86e7b2a5bce7 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:07.639414 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af793503-0626-42cf-a100-86e7b2a5bce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:07.639689 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af793503-0626-42cf-a100-86e7b2a5bce7 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:07.640290 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 633/1261] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:07.640401 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af793503-0626-42cf-a100-86e7b2a5bce7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:07.641328 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a42d6-6c59-4137-a840-29c9389dcc6f None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:07.641607 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a42d6-6c59-4137-a840-29c9389dcc6f None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:07.641881 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869a42d6-6c59-4137-a840-29c9389dcc6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:07.642237 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af793503-0626-42cf-a100-86e7b2a5bce7 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:07.661238 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-869a42d6-6c59-4137-a840-29c9389dcc6f None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:07.662478 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-869a42d6-6c59-4137-a840-29c9389dcc6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:07.663995 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 629/1262] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:05:07 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:07.664800 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af793503-0626-42cf-a100-86e7b2a5bce7 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:07.670117 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 634/1263] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:07.670816 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a42d6-6c59-4137-a840-29c9389dcc6f None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:07.678422 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af793503-0626-42cf-a100-86e7b2a5bce7 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:07.680269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869a42d6-6c59-4137-a840-29c9389dcc6f None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:07.683946 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c59e5f-094d-4575-96a6-c60af9e2c7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:07.684531 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c59e5f-094d-4575-96a6-c60af9e2c7ac admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:07.684817 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c59e5f-094d-4575-96a6-c60af9e2c7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:07.684962 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc54ea-7d7e-4bde-b984-40b09fa9db00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:07.685094 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c59e5f-094d-4575-96a6-c60af9e2c7ac admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:07.685413 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc54ea-7d7e-4bde-b984-40b09fa9db00 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:07.685723 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc54ea-7d7e-4bde-b984-40b09fa9db00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:07.685846 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c59e5f-094d-4575-96a6-c60af9e2c7ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:07.685994 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc54ea-7d7e-4bde-b984-40b09fa9db00 admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:07.686648 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc54ea-7d7e-4bde-b984-40b09fa9db00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:07.687671 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c59e5f-094d-4575-96a6-c60af9e2c7ac admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:07.688514 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcc54ea-7d7e-4bde-b984-40b09fa9db00 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:07.712666 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 630/1264] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:05:07 2022] POST /identity/v3/users => generated 287 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:07.712666 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c59e5f-094d-4575-96a6-c60af9e2c7ac admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:07.712666 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-37c59e5f-094d-4575-96a6-c60af9e2c7ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:07.714003 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 635/1265] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:05:07 2022] POST /identity/v3/users => generated 292 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:07.714793 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc54ea-7d7e-4bde-b984-40b09fa9db00 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:07.725024 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc54ea-7d7e-4bde-b984-40b09fa9db00 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:07.730394 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee892c-02ce-4157-8adc-14bf8cea6915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:07.731140 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee892c-02ce-4157-8adc-14bf8cea6915 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:07.731260 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee892c-02ce-4157-8adc-14bf8cea6915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:07.731523 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ee892c-02ce-4157-8adc-14bf8cea6915 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:07.732163 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ee892c-02ce-4157-8adc-14bf8cea6915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:07.733737 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ee892c-02ce-4157-8adc-14bf8cea6915 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:07.748571 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 636/1266] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:07.749436 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee892c-02ce-4157-8adc-14bf8cea6915 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:07.759079 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ee892c-02ce-4157-8adc-14bf8cea6915 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:07.764317 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca88a8-be5c-4068-a9d9-4c2490cf9e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:07.764900 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca88a8-be5c-4068-a9d9-4c2490cf9e46 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:07.765183 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca88a8-be5c-4068-a9d9-4c2490cf9e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:07.765449 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ca88a8-be5c-4068-a9d9-4c2490cf9e46 admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:07.766064 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca88a8-be5c-4068-a9d9-4c2490cf9e46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:07.767991 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ca88a8-be5c-4068-a9d9-4c2490cf9e46 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:07.772773 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c59e5f-094d-4575-96a6-c60af9e2c7ac admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:07.776463 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040ee5d-92fa-4278-865c-6c53c9606e61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:07.777079 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040ee5d-92fa-4278-865c-6c53c9606e61 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:07.777358 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040ee5d-92fa-4278-865c-6c53c9606e61 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:07.777638 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f040ee5d-92fa-4278-865c-6c53c9606e61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:07.788024 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f040ee5d-92fa-4278-865c-6c53c9606e61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:07.789641 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 637/1267] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:05:07 2022] POST /identity/v3/users => generated 301 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:07.790036 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f040ee5d-92fa-4278-865c-6c53c9606e61 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:07.790595 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca88a8-be5c-4068-a9d9-4c2490cf9e46 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:07.799981 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 631/1268] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:05:07.800325 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ca88a8-be5c-4068-a9d9-4c2490cf9e46 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:07.800766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040ee5d-92fa-4278-865c-6c53c9606e61 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:07.805044 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c73f1a-9dcd-4ef0-8294-d7a5e518c013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:07.805692 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c73f1a-9dcd-4ef0-8294-d7a5e518c013 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:07.805980 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c73f1a-9dcd-4ef0-8294-d7a5e518c013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:07.806372 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c73f1a-9dcd-4ef0-8294-d7a5e518c013 admin admin] PATH_INFO: `/v3/projects/67e002b62fa7450b809a03d93496e7f1/users/8b4d77a96af94f71b76c7b00390b56a3/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:07.809811 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f040ee5d-92fa-4278-865c-6c53c9606e61 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:07.814725 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c67a66-fe6b-4c60-8f13-05706ba25a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:07.815364 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c67a66-fe6b-4c60-8f13-05706ba25a5f admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:07.815643 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c67a66-fe6b-4c60-8f13-05706ba25a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:07.816416 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c67a66-fe6b-4c60-8f13-05706ba25a5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:07.816619 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c67a66-fe6b-4c60-8f13-05706ba25a5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:07.818308 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c67a66-fe6b-4c60-8f13-05706ba25a5f admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:07.822517 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c73f1a-9dcd-4ef0-8294-d7a5e518c013 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67e002b62fa7450b809a03d93496e7f1, user_id=8b4d77a96af94f71b76c7b00390b56a3, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:07.825721 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c73f1a-9dcd-4ef0-8294-d7a5e518c013 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:07.830529 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 632/1269] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:07.831484 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c67a66-fe6b-4c60-8f13-05706ba25a5f admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:07.851884 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 638/1270] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:07 2022] PUT /identity/v3/projects/67e002b62fa7450b809a03d93496e7f1/users/8b4d77a96af94f71b76c7b00390b56a3/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:07.852051 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c67a66-fe6b-4c60-8f13-05706ba25a5f admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:07.852152 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c73f1a-9dcd-4ef0-8294-d7a5e518c013 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:07.906539 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb861d-c37c-47df-8216-637cca4a4fe9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:07.907158 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb861d-c37c-47df-8216-637cca4a4fe9 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:07.907507 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb861d-c37c-47df-8216-637cca4a4fe9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:07.907745 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eb861d-c37c-47df-8216-637cca4a4fe9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:07.912186 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c73f1a-9dcd-4ef0-8294-d7a5e518c013 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:07.917166 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ff077-7081-4011-92ab-4233d23623fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:07.917794 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ff077-7081-4011-92ab-4233d23623fd admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:07.918077 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ff077-7081-4011-92ab-4233d23623fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:07.918387 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7ff077-7081-4011-92ab-4233d23623fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:07.919212 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb861d-c37c-47df-8216-637cca4a4fe9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:07.919685 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7ff077-7081-4011-92ab-4233d23623fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:07.920865 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eb861d-c37c-47df-8216-637cca4a4fe9 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:07.921303 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7ff077-7081-4011-92ab-4233d23623fd admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:07.934518 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 639/1271] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:07.934518 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ff077-7081-4011-92ab-4233d23623fd admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:07.934819 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 633/1272] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:05:07.934819 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb861d-c37c-47df-8216-637cca4a4fe9 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:07.945114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eb861d-c37c-47df-8216-637cca4a4fe9 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:07.945790 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7ff077-7081-4011-92ab-4233d23623fd admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:07.951029 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b78d10-5cb9-43e8-9281-91f9fff032fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:07.951450 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf3a438-a606-4c5a-b737-44ea81323e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:07.951855 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b78d10-5cb9-43e8-9281-91f9fff032fa admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:07.952198 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b78d10-5cb9-43e8-9281-91f9fff032fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:07.952314 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3a438-a606-4c5a-b737-44ea81323e54 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:07.952592 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b78d10-5cb9-43e8-9281-91f9fff032fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:07.952832 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3a438-a606-4c5a-b737-44ea81323e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:07.953174 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf3a438-a606-4c5a-b737-44ea81323e54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:07.953450 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b78d10-5cb9-43e8-9281-91f9fff032fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:07.953954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf3a438-a606-4c5a-b737-44ea81323e54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:07.955222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b78d10-5cb9-43e8-9281-91f9fff032fa admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:07.955726 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf3a438-a606-4c5a-b737-44ea81323e54 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:07.969379 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 640/1273] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:07.969379 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b78d10-5cb9-43e8-9281-91f9fff032fa admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:07.969793 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 634/1274] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:07.970490 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf3a438-a606-4c5a-b737-44ea81323e54 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:07.979210 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:07.982071 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf3a438-a606-4c5a-b737-44ea81323e54 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:07.984286 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b78d10-5cb9-43e8-9281-91f9fff032fa admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:07.990016 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04f08b-eeab-43f0-bdd8-504afef1903a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:07.991385 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04f08b-eeab-43f0-bdd8-504afef1903a admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:07.991773 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04f08b-eeab-43f0-bdd8-504afef1903a admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:07.992235 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb04f08b-eeab-43f0-bdd8-504afef1903a admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:07.992427 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20735018-82f7-4224-8ae2-7662f7d38ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:07.993170 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20735018-82f7-4224-8ae2-7662f7d38ed0 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:07.993300 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04f08b-eeab-43f0-bdd8-504afef1903a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:07.993617 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20735018-82f7-4224-8ae2-7662f7d38ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:07.993959 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20735018-82f7-4224-8ae2-7662f7d38ed0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:07.994926 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20735018-82f7-4224-8ae2-7662f7d38ed0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:07.995392 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb04f08b-eeab-43f0-bdd8-504afef1903a admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:07.996665 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20735018-82f7-4224-8ae2-7662f7d38ed0 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:08.008893 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 635/1275] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:08.010495 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb572b-f51f-4116-a36d-51024df97f0b None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:08.011070 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb572b-f51f-4116-a36d-51024df97f0b None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:08.011482 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fb572b-f51f-4116-a36d-51024df97f0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:08.016274 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 641/1276] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:07 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:08.017303 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04f08b-eeab-43f0-bdd8-504afef1903a admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:08.034380 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb04f08b-eeab-43f0-bdd8-504afef1903a admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:08.039777 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda671e-1a4a-47e4-a7a1-6bb2e56995c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:08.040472 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda671e-1a4a-47e4-a7a1-6bb2e56995c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:08.040827 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda671e-1a4a-47e4-a7a1-6bb2e56995c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:08.041175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bda671e-1a4a-47e4-a7a1-6bb2e56995c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:08.041899 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bda671e-1a4a-47e4-a7a1-6bb2e56995c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:08.043758 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bda671e-1a4a-47e4-a7a1-6bb2e56995c6 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:08.047308 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-95fb572b-f51f-4116-a36d-51024df97f0b None None] MFA Rules not processed for user `8b4d77a96af94f71b76c7b00390b56a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:08.048688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-95fb572b-f51f-4116-a36d-51024df97f0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:08.058149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 642/1277] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:08.058149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda671e-1a4a-47e4-a7a1-6bb2e56995c6 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:08.067607 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bda671e-1a4a-47e4-a7a1-6bb2e56995c6 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:08.073139 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe93810-6455-475f-922e-f468a70aa02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:08.074309 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe93810-6455-475f-922e-f468a70aa02b admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:08.075113 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe93810-6455-475f-922e-f468a70aa02b admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:08.075113 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe93810-6455-475f-922e-f468a70aa02b admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:08.075740 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe93810-6455-475f-922e-f468a70aa02b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:08.077425 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe93810-6455-475f-922e-f468a70aa02b admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:08.089588 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 643/1278] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:08.090517 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe93810-6455-475f-922e-f468a70aa02b admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:08.104446 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe93810-6455-475f-922e-f468a70aa02b admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:08.109533 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a83f7e-5cc7-4b18-8738-9d33f79ff6cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:08.110228 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a83f7e-5cc7-4b18-8738-9d33f79ff6cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:08.110544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a83f7e-5cc7-4b18-8738-9d33f79ff6cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:08.110822 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a83f7e-5cc7-4b18-8738-9d33f79ff6cc admin admin] PATH_INFO: `/v3/projects/e41c21bcf4ca457391486c08565b2aae/users/4b6a101d23df4b62ba7e0c35d28d2a3c/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:08.128898 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a83f7e-5cc7-4b18-8738-9d33f79ff6cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=e41c21bcf4ca457391486c08565b2aae, user_id=4b6a101d23df4b62ba7e0c35d28d2a3c, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:08.131671 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a83f7e-5cc7-4b18-8738-9d33f79ff6cc admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:08.132630 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 636/1279] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:08.136531 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:08.138420 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb572b-f51f-4116-a36d-51024df97f0b None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:08.154094 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fb572b-f51f-4116-a36d-51024df97f0b None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:08.160978 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 644/1280] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:08 2022] PUT /identity/v3/projects/e41c21bcf4ca457391486c08565b2aae/users/4b6a101d23df4b62ba7e0c35d28d2a3c/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:08.160978 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a83f7e-5cc7-4b18-8738-9d33f79ff6cc admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:08.248537 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d9b20-3fcf-48bb-be07-bc8ad28ed7ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:08.250551 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d9b20-3fcf-48bb-be07-bc8ad28ed7ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:08.251011 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d9b20-3fcf-48bb-be07-bc8ad28ed7ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:08.251480 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d9b20-3fcf-48bb-be07-bc8ad28ed7ae admin admin] PATH_INFO: `/v3/projects/5ebb0d7d16214912a585ca96cf6b2c6a/users/7aca00ea597140c5be97a6a4fa33f9c7/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:08.271834 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d9b20-3fcf-48bb-be07-bc8ad28ed7ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ebb0d7d16214912a585ca96cf6b2c6a, user_id=7aca00ea597140c5be97a6a4fa33f9c7, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:08.276039 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5d9b20-3fcf-48bb-be07-bc8ad28ed7ae admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:08.300770 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 637/1281] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:08 2022] PUT /identity/v3/projects/5ebb0d7d16214912a585ca96cf6b2c6a/users/7aca00ea597140c5be97a6a4fa33f9c7/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:08.304116 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d9b20-3fcf-48bb-be07-bc8ad28ed7ae admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:08.304695 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a83f7e-5cc7-4b18-8738-9d33f79ff6cc admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:08.398068 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d9b20-3fcf-48bb-be07-bc8ad28ed7ae admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:08.405276 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c8d202-f5c6-41e8-9818-52685daa3380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:08.406151 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8d202-f5c6-41e8-9818-52685daa3380 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:08.406462 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8d202-f5c6-41e8-9818-52685daa3380 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:08.406736 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8d202-f5c6-41e8-9818-52685daa3380 admin admin] PATH_INFO: `/v3/projects/35f8e87c854e4587884430909632cad4/users/3595cfbefb134e27a69dbefa059fc1b1/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:08.420254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:08.421295 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18c492-b0c4-4a60-af19-85ae45779023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:08.422484 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18c492-b0c4-4a60-af19-85ae45779023 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:08.422822 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18c492-b0c4-4a60-af19-85ae45779023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:08.423118 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18c492-b0c4-4a60-af19-85ae45779023 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:08.424129 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c18c492-b0c4-4a60-af19-85ae45779023 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:08.425931 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c18c492-b0c4-4a60-af19-85ae45779023 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:08.426150 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c8d202-f5c6-41e8-9818-52685daa3380 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35f8e87c854e4587884430909632cad4, user_id=3595cfbefb134e27a69dbefa059fc1b1, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:08.430072 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c8d202-f5c6-41e8-9818-52685daa3380 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:08.440156 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 645/1282] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:08 2022] GET /identity/v3/roles => generated 1420 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:08.444653 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18c492-b0c4-4a60-af19-85ae45779023 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:08.461341 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 638/1283] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:08 2022] PUT /identity/v3/projects/35f8e87c854e4587884430909632cad4/users/3595cfbefb134e27a69dbefa059fc1b1/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:08.461341 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f803f-810c-4fe1-97e8-24467d12fa21 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:08.461341 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f803f-810c-4fe1-97e8-24467d12fa21 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:08.461341 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5f803f-810c-4fe1-97e8-24467d12fa21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:08.465808 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18c492-b0c4-4a60-af19-85ae45779023 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:08.485303 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-0b5f803f-810c-4fe1-97e8-24467d12fa21 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:08.486681 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-0b5f803f-810c-4fe1-97e8-24467d12fa21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:08.582929 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447c4d5-6661-4394-abf6-fdefc97eefe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:08.584114 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447c4d5-6661-4394-abf6-fdefc97eefe6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:08.584487 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447c4d5-6661-4394-abf6-fdefc97eefe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:08.584804 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9447c4d5-6661-4394-abf6-fdefc97eefe6 admin admin] PATH_INFO: `/v3/projects/9ba293d8c83844d28dd78b7d14c66771/users/8c82afa51f3a482ba72051e95fac2713/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:08.609732 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9447c4d5-6661-4394-abf6-fdefc97eefe6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ba293d8c83844d28dd78b7d14c66771, user_id=8c82afa51f3a482ba72051e95fac2713, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:08.613325 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 639/1284] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:08.614273 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9447c4d5-6661-4394-abf6-fdefc97eefe6 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:08.616804 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40ae16-464a-49d3-ac7c-40e95cc15117 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:08.620483 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40ae16-464a-49d3-ac7c-40e95cc15117 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:08.620956 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40ae16-464a-49d3-ac7c-40e95cc15117 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:08.622676 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 640/1285] 10.222.0.13 () {54 vars in 961 bytes} [Sun Oct 23 20:05:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:05:08.626014 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a11ea-ed80-4068-a04c-2eb032dfbf51 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:08.626559 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a11ea-ed80-4068-a04c-2eb032dfbf51 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:08.628172 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7a11ea-ed80-4068-a04c-2eb032dfbf51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:08.645160 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 646/1286] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:08 2022] PUT /identity/v3/projects/9ba293d8c83844d28dd78b7d14c66771/users/8c82afa51f3a482ba72051e95fac2713/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:08.645160 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447c4d5-6661-4394-abf6-fdefc97eefe6 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:08.665066 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-be7a11ea-ed80-4068-a04c-2eb032dfbf51 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:08.667322 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-be7a11ea-ed80-4068-a04c-2eb032dfbf51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:08.671326 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:08.750396 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 641/1287] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:08.750396 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b3838f-c724-4816-bec3-19870ae5f245 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:08.750396 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b3838f-c724-4816-bec3-19870ae5f245 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:08.750396 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b3838f-c724-4816-bec3-19870ae5f245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:08.753469 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9447c4d5-6661-4394-abf6-fdefc97eefe6 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:08.765716 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28709d3e-9bfd-4a14-abb1-aef1c6929267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:08.766835 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28709d3e-9bfd-4a14-abb1-aef1c6929267 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:08.767277 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28709d3e-9bfd-4a14-abb1-aef1c6929267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:08.767712 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28709d3e-9bfd-4a14-abb1-aef1c6929267 admin admin] PATH_INFO: `/v3/projects/ca2ed1e33c3b40a6b5c7aad91cffdbc7/users/9a3030e91531434f9bba4bb518ebf424/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:08.787759 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-58b3838f-c724-4816-bec3-19870ae5f245 None None] MFA Rules not processed for user `3595cfbefb134e27a69dbefa059fc1b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:08.791387 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-58b3838f-c724-4816-bec3-19870ae5f245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:08.802099 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28709d3e-9bfd-4a14-abb1-aef1c6929267 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca2ed1e33c3b40a6b5c7aad91cffdbc7, user_id=9a3030e91531434f9bba4bb518ebf424, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:08.813244 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28709d3e-9bfd-4a14-abb1-aef1c6929267 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:08.840060 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 647/1288] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:08 2022] PUT /identity/v3/projects/ca2ed1e33c3b40a6b5c7aad91cffdbc7/users/9a3030e91531434f9bba4bb518ebf424/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:08.840992 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28709d3e-9bfd-4a14-abb1-aef1c6929267 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:08.904067 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 642/1289] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:08 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:08.905655 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2fd01-29ac-49b6-8fcd-a52d5343300e None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:08.906963 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2fd01-29ac-49b6-8fcd-a52d5343300e None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:08.907438 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e2fd01-29ac-49b6-8fcd-a52d5343300e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:08.950799 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28709d3e-9bfd-4a14-abb1-aef1c6929267 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:08.964119 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd46498-a773-49b7-8a58-0bd449026340 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:08.965148 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd46498-a773-49b7-8a58-0bd449026340 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:08.965723 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd46498-a773-49b7-8a58-0bd449026340 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:08.966306 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd46498-a773-49b7-8a58-0bd449026340 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:08.977785 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:08.987026 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-89e2fd01-29ac-49b6-8fcd-a52d5343300e None None] MFA Rules not processed for user `0b3d512304254ffcaa8dad90abf1f9c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:08.989292 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-89e2fd01-29ac-49b6-8fcd-a52d5343300e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:09.087679 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd46498-a773-49b7-8a58-0bd449026340 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:09.090763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd46498-a773-49b7-8a58-0bd449026340 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:09.117040 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 648/1290] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:05:09.118762 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e8278-f8d7-43f3-92ff-eb22f5a933b2 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:09.121781 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e8278-f8d7-43f3-92ff-eb22f5a933b2 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:09.124019 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e8278-f8d7-43f3-92ff-eb22f5a933b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:09.136941 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 643/1291] 10.222.0.13 () {58 vars in 1058 bytes} [Sun Oct 23 20:05:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:09.138595 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2fd01-29ac-49b6-8fcd-a52d5343300e None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:09.158601 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e2fd01-29ac-49b6-8fcd-a52d5343300e None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:09.172337 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a613e1a8-2e5b-49c1-bfdb-5181a4e70391 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:09.174240 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613e1a8-2e5b-49c1-bfdb-5181a4e70391 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:09.174909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613e1a8-2e5b-49c1-bfdb-5181a4e70391 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:09.175036 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-9b1e8278-f8d7-43f3-92ff-eb22f5a933b2 None None] MFA Rules not processed for user `8c82afa51f3a482ba72051e95fac2713`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:09.175511 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613e1a8-2e5b-49c1-bfdb-5181a4e70391 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:09.176302 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-9b1e8278-f8d7-43f3-92ff-eb22f5a933b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:09.195655 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a613e1a8-2e5b-49c1-bfdb-5181a4e70391 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:09.198884 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a613e1a8-2e5b-49c1-bfdb-5181a4e70391 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:09.219362 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 644/1292] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:05:09.224824 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b55e9b-fac8-4d0e-99e6-69aaaef00ffc None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:09.225159 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b55e9b-fac8-4d0e-99e6-69aaaef00ffc None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:09.225447 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b55e9b-fac8-4d0e-99e6-69aaaef00ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:09.236315 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:09.271407 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 649/1293] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:09.271407 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e8278-f8d7-43f3-92ff-eb22f5a933b2 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:09.282010 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-f4b55e9b-fac8-4d0e-99e6-69aaaef00ffc None None] MFA Rules not processed for user `9a3030e91531434f9bba4bb518ebf424`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:09.285188 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f4b55e9b-fac8-4d0e-99e6-69aaaef00ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:09.293589 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e8278-f8d7-43f3-92ff-eb22f5a933b2 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:09.299677 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c0da0-4b4c-49c5-9a1a-41ff9822937d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:09.300460 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c0da0-4b4c-49c5-9a1a-41ff9822937d admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:09.300894 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c0da0-4b4c-49c5-9a1a-41ff9822937d admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:09.301192 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c0da0-4b4c-49c5-9a1a-41ff9822937d admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:09.301945 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294c0da0-4b4c-49c5-9a1a-41ff9822937d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:09.304976 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294c0da0-4b4c-49c5-9a1a-41ff9822937d admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:09.349012 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 650/1294] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:05:09 2022] POST /identity/v3/projects => generated 375 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:09.350758 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c0da0-4b4c-49c5-9a1a-41ff9822937d admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:09.396798 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 645/1295] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:09 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:09.399099 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9785f0-e880-4bc7-a534-937ed665b343 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:09.399526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9785f0-e880-4bc7-a534-937ed665b343 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:09.399992 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9785f0-e880-4bc7-a534-937ed665b343 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:09.402673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 646/1296] 10.222.0.13 () {54 vars in 961 bytes} [Sun Oct 23 20:05:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:05:09.404961 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9785f0-e880-4bc7-a534-937ed665b343 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:09.516236 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c0da0-4b4c-49c5-9a1a-41ff9822937d admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:09.516946 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9785f0-e880-4bc7-a534-937ed665b343 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:09.527586 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08b54e-1ee2-4816-b973-46340435eed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:09.528312 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08b54e-1ee2-4816-b973-46340435eed5 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:09.528635 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08b54e-1ee2-4816-b973-46340435eed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:09.528910 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf08b54e-1ee2-4816-b973-46340435eed5 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:09.530134 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf08b54e-1ee2-4816-b973-46340435eed5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:09.534368 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf08b54e-1ee2-4816-b973-46340435eed5 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:09.534490 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a277f1e8-126a-40e6-9e1b-ace6c21547ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:09.535256 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277f1e8-126a-40e6-9e1b-ace6c21547ba service nova] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:09.535676 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277f1e8-126a-40e6-9e1b-ace6c21547ba service nova] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:09.535994 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a277f1e8-126a-40e6-9e1b-ace6c21547ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:09.589393 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 647/1297] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:05:09 2022] POST /identity/v3/users => generated 294 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:09.591307 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08b54e-1ee2-4816-b973-46340435eed5 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:09.601759 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:09.606028 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf08b54e-1ee2-4816-b973-46340435eed5 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:09.612802 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5daf43-b9b1-45c1-8b40-035ccfba0708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:09.613528 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5daf43-b9b1-45c1-8b40-035ccfba0708 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:09.613860 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5daf43-b9b1-45c1-8b40-035ccfba0708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:09.614463 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5daf43-b9b1-45c1-8b40-035ccfba0708 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:09.615314 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5daf43-b9b1-45c1-8b40-035ccfba0708 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:09.618510 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5daf43-b9b1-45c1-8b40-035ccfba0708 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:09.658441 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 648/1298] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:05:09 2022] POST /identity/v3/projects => generated 399 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:09.660432 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffc17f-655b-4bd4-9d80-67df7ee56326 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:09.660712 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffc17f-655b-4bd4-9d80-67df7ee56326 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:09.661163 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ffc17f-655b-4bd4-9d80-67df7ee56326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:09.679451 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a277f1e8-126a-40e6-9e1b-ace6c21547ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:09.681759 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a277f1e8-126a-40e6-9e1b-ace6c21547ba service nova] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:09.690062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-65ffc17f-655b-4bd4-9d80-67df7ee56326 None None] MFA Rules not processed for user `0b3d512304254ffcaa8dad90abf1f9c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:09.691529 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-65ffc17f-655b-4bd4-9d80-67df7ee56326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:09.705568 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 651/1299] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:09 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:05:09.706649 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a277f1e8-126a-40e6-9e1b-ace6c21547ba service nova] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:09.733284 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a277f1e8-126a-40e6-9e1b-ace6c21547ba service nova] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:09.879243 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d350b63-252b-49e5-94a5-5dc35cce29a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:09.879875 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d350b63-252b-49e5-94a5-5dc35cce29a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:09.880151 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d350b63-252b-49e5-94a5-5dc35cce29a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:09.880417 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d350b63-252b-49e5-94a5-5dc35cce29a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:09.881062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 649/1300] 10.222.0.13 () {58 vars in 1058 bytes} [Sun Oct 23 20:05:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:09.881254 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d350b63-252b-49e5-94a5-5dc35cce29a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:09.885334 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d350b63-252b-49e5-94a5-5dc35cce29a1 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:09.889271 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:09.892172 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ffc17f-655b-4bd4-9d80-67df7ee56326 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:09.914358 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 652/1301] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:09 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:09.915528 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d350b63-252b-49e5-94a5-5dc35cce29a1 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:09.920495 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ffc17f-655b-4bd4-9d80-67df7ee56326 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:09.926625 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b59ab5-04a2-4fb0-be88-cc4ed3b2bb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:09.927256 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b59ab5-04a2-4fb0-be88-cc4ed3b2bb3a admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:09.927536 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b59ab5-04a2-4fb0-be88-cc4ed3b2bb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:09.927808 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b59ab5-04a2-4fb0-be88-cc4ed3b2bb3a admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:09.928454 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b59ab5-04a2-4fb0-be88-cc4ed3b2bb3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:09.935949 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b59ab5-04a2-4fb0-be88-cc4ed3b2bb3a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:09.948330 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d350b63-252b-49e5-94a5-5dc35cce29a1 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:09.963067 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d1975-caf7-4087-9dd2-9a742453e511 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:09.963963 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d1975-caf7-4087-9dd2-9a742453e511 service nova] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:09.964615 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d1975-caf7-4087-9dd2-9a742453e511 service nova] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:09.965039 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d1975-caf7-4087-9dd2-9a742453e511 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:09.993249 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 650/1302] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:05:09 2022] POST /identity/v3/users => generated 306 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:09.994169 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b59ab5-04a2-4fb0-be88-cc4ed3b2bb3a admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:10.023453 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b59ab5-04a2-4fb0-be88-cc4ed3b2bb3a admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:10.038203 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268893d-4619-4f61-9282-6709c3f12b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:10.039482 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268893d-4619-4f61-9282-6709c3f12b68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:10.039768 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268893d-4619-4f61-9282-6709c3f12b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:10.040048 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268893d-4619-4f61-9282-6709c3f12b68 admin admin] PATH_INFO: `/v3/projects/75b1c8bae8d24e92a04d48dbebd9fcf5/users/bf1bfc78bfff415d818a8ef9baf5a06d/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:10.074393 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268893d-4619-4f61-9282-6709c3f12b68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75b1c8bae8d24e92a04d48dbebd9fcf5, user_id=bf1bfc78bfff415d818a8ef9baf5a06d, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:10.077205 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268893d-4619-4f61-9282-6709c3f12b68 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:10.107753 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 651/1303] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:09 2022] PUT /identity/v3/projects/75b1c8bae8d24e92a04d48dbebd9fcf5/users/bf1bfc78bfff415d818a8ef9baf5a06d/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:10.109196 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268893d-4619-4f61-9282-6709c3f12b68 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:10.124581 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40d1975-caf7-4087-9dd2-9a742453e511 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:10.126645 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40d1975-caf7-4087-9dd2-9a742453e511 service nova] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:10.153149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 653/1304] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:05:10.154319 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d1975-caf7-4087-9dd2-9a742453e511 service nova] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:10.179639 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d1975-caf7-4087-9dd2-9a742453e511 service nova] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:10.231147 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268893d-4619-4f61-9282-6709c3f12b68 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:10.237064 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e68438-c3cd-4488-975e-aaafb26b6f1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:10.238023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e68438-c3cd-4488-975e-aaafb26b6f1b admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:10.238344 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e68438-c3cd-4488-975e-aaafb26b6f1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:10.238968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e68438-c3cd-4488-975e-aaafb26b6f1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:10.239500 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e68438-c3cd-4488-975e-aaafb26b6f1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:10.241339 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e68438-c3cd-4488-975e-aaafb26b6f1b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:10.253793 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 652/1305] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:10.257473 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e68438-c3cd-4488-975e-aaafb26b6f1b admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:10.286439 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e68438-c3cd-4488-975e-aaafb26b6f1b admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:10.298363 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c942a9-3147-430d-9f6d-d4ef609301a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:10.299275 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c942a9-3147-430d-9f6d-d4ef609301a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:10.302678 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c942a9-3147-430d-9f6d-d4ef609301a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:10.303335 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c942a9-3147-430d-9f6d-d4ef609301a5 admin admin] PATH_INFO: `/v3/projects/daa4d2797ae545b78d9e7daefe273410/users/a9ee4775d841485f8f38313bf5a90c0e/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:10.346968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c942a9-3147-430d-9f6d-d4ef609301a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daa4d2797ae545b78d9e7daefe273410, user_id=a9ee4775d841485f8f38313bf5a90c0e, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:10.353137 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c942a9-3147-430d-9f6d-d4ef609301a5 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:10.361624 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17bbbdc0-3d91-4270-a69e-26a005ccee48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:10.362709 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbbdc0-3d91-4270-a69e-26a005ccee48 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:10.362709 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbbdc0-3d91-4270-a69e-26a005ccee48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:10.362925 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17bbbdc0-3d91-4270-a69e-26a005ccee48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:10.363530 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bbbdc0-3d91-4270-a69e-26a005ccee48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:10.365201 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17bbbdc0-3d91-4270-a69e-26a005ccee48 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:10.376940 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 653/1306] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:10 2022] PUT /identity/v3/projects/daa4d2797ae545b78d9e7daefe273410/users/a9ee4775d841485f8f38313bf5a90c0e/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:10.378864 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0feb335-e72a-4049-8857-7fc85d329bc0 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:10.379455 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0feb335-e72a-4049-8857-7fc85d329bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:10.379828 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0feb335-e72a-4049-8857-7fc85d329bc0 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:10.386811 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 654/1307] 10.222.0.13 () {54 vars in 912 bytes} [Sun Oct 23 20:05:10 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:05:10.387804 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0feb335-e72a-4049-8857-7fc85d329bc0 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:10.394757 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:10.405298 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 654/1308] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:10.408084 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9961b9c-a6ff-41a9-94ae-6aff34974585 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:10.414890 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9961b9c-a6ff-41a9-94ae-6aff34974585 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:10.415164 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9961b9c-a6ff-41a9-94ae-6aff34974585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:10.529823 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-b9961b9c-a6ff-41a9-94ae-6aff34974585 None None] MFA Rules not processed for user `0b3d512304254ffcaa8dad90abf1f9c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:10.532567 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0feb335-e72a-4049-8857-7fc85d329bc0 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:10.533001 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-b9961b9c-a6ff-41a9-94ae-6aff34974585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:10.533093 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6966a14-1e21-479e-be0f-44c48667c584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:10.533093 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6966a14-1e21-479e-be0f-44c48667c584 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:10.533093 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6966a14-1e21-479e-be0f-44c48667c584 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:10.533093 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6966a14-1e21-479e-be0f-44c48667c584 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:10.533384 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6966a14-1e21-479e-be0f-44c48667c584 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:10.533384 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6966a14-1e21-479e-be0f-44c48667c584 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:10.533384 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 655/1309] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:10 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:10.533384 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6966a14-1e21-479e-be0f-44c48667c584 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:10.549406 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6966a14-1e21-479e-be0f-44c48667c584 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:10.564756 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4016285-fc10-4c58-9e7f-8a0835c92f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:10.575744 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4016285-fc10-4c58-9e7f-8a0835c92f17 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:10.576146 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4016285-fc10-4c58-9e7f-8a0835c92f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:10.577683 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4016285-fc10-4c58-9e7f-8a0835c92f17 admin admin] PATH_INFO: `/v3/projects/75b1c8bae8d24e92a04d48dbebd9fcf5/users/bf1bfc78bfff415d818a8ef9baf5a06d/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:10.582603 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4016285-fc10-4c58-9e7f-8a0835c92f17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75b1c8bae8d24e92a04d48dbebd9fcf5, user_id=bf1bfc78bfff415d818a8ef9baf5a06d, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:10.592801 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4016285-fc10-4c58-9e7f-8a0835c92f17 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:10.633446 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 656/1310] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:10 2022] PUT /identity/v3/projects/75b1c8bae8d24e92a04d48dbebd9fcf5/users/bf1bfc78bfff415d818a8ef9baf5a06d/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:10.634580 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4016285-fc10-4c58-9e7f-8a0835c92f17 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:10.683156 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 655/1311] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Oct 23 20:05:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:10.684663 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a560c-2444-40d3-8497-01d84e86bc87 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:10.684911 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a560c-2444-40d3-8497-01d84e86bc87 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:10.685230 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265a560c-2444-40d3-8497-01d84e86bc87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:10.747620 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-265a560c-2444-40d3-8497-01d84e86bc87 None None] MFA Rules not processed for user `bf1bfc78bfff415d818a8ef9baf5a06d`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:10.748856 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-265a560c-2444-40d3-8497-01d84e86bc87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:10.765372 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4016285-fc10-4c58-9e7f-8a0835c92f17 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:10.771007 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:10.777656 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b375d73-b348-48e8-a795-378046b827a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:10.778376 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b375d73-b348-48e8-a795-378046b827a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:10.778663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b375d73-b348-48e8-a795-378046b827a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:10.779232 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b375d73-b348-48e8-a795-378046b827a8 admin admin] PATH_INFO: `/v3/projects/daa4d2797ae545b78d9e7daefe273410/users/a9ee4775d841485f8f38313bf5a90c0e/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:10.781103 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b375d73-b348-48e8-a795-378046b827a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daa4d2797ae545b78d9e7daefe273410, user_id=a9ee4775d841485f8f38313bf5a90c0e, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:10.783895 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b375d73-b348-48e8-a795-378046b827a8 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:10.799110 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 657/1312] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:10 2022] PUT /identity/v3/projects/daa4d2797ae545b78d9e7daefe273410/users/a9ee4775d841485f8f38313bf5a90c0e/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:10.800129 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b375d73-b348-48e8-a795-378046b827a8 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:10.893489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 656/1313] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:10 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:10.897274 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175ff04-3319-497d-8a40-4c1192cbcf42 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:10.897940 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175ff04-3319-497d-8a40-4c1192cbcf42 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:10.898605 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e175ff04-3319-497d-8a40-4c1192cbcf42 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:10.901415 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 657/1314] 10.222.0.13 () {54 vars in 912 bytes} [Sun Oct 23 20:05:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:05:10.903021 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324773b-00ee-4925-a2d3-0108380be829 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:10.903776 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324773b-00ee-4925-a2d3-0108380be829 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:10.904742 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3324773b-00ee-4925-a2d3-0108380be829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:10.952074 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b375d73-b348-48e8-a795-378046b827a8 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:10.960884 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef8e21-6e6a-4b2d-a0f2-6be8c58434b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:10.963982 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-3324773b-00ee-4925-a2d3-0108380be829 None None] MFA Rules not processed for user `a9ee4775d841485f8f38313bf5a90c0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:10.965934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-3324773b-00ee-4925-a2d3-0108380be829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:10.975167 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef8e21-6e6a-4b2d-a0f2-6be8c58434b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:10.975457 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef8e21-6e6a-4b2d-a0f2-6be8c58434b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:10.977185 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef8e21-6e6a-4b2d-a0f2-6be8c58434b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:11.091100 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:11.246800 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 658/1315] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:10 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:11.247879 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3324773b-00ee-4925-a2d3-0108380be829 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:11.249821 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef8e21-6e6a-4b2d-a0f2-6be8c58434b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:11.252109 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef8e21-6e6a-4b2d-a0f2-6be8c58434b1 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:11.277414 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 658/1316] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:05:11.279016 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2ff33-7272-496e-82ca-bd2c62527a6c None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:11.279690 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2ff33-7272-496e-82ca-bd2c62527a6c None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:11.279690 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb2ff33-7272-496e-82ca-bd2c62527a6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:11.288182 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3324773b-00ee-4925-a2d3-0108380be829 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:11.306183 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31974bcb-2db4-4858-8a2a-4be3a3e21d90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:11.307688 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31974bcb-2db4-4858-8a2a-4be3a3e21d90 service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:11.308103 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31974bcb-2db4-4858-8a2a-4be3a3e21d90 service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:11.314993 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31974bcb-2db4-4858-8a2a-4be3a3e21d90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:11.330279 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-adb2ff33-7272-496e-82ca-bd2c62527a6c None None] MFA Rules not processed for user `0b3d512304254ffcaa8dad90abf1f9c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:11.332995 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-adb2ff33-7272-496e-82ca-bd2c62527a6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:11.363300 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 659/1317] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Oct 23 20:05:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:11.442812 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31974bcb-2db4-4858-8a2a-4be3a3e21d90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:11.444605 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31974bcb-2db4-4858-8a2a-4be3a3e21d90 service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:11.460015 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 659/1318] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:05:12.296686 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2ff33-7272-496e-82ca-bd2c62527a6c None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:12.321202 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb2ff33-7272-496e-82ca-bd2c62527a6c None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:12.338412 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64881ba9-4958-40c0-8970-44ce5731316e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:12.339976 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64881ba9-4958-40c0-8970-44ce5731316e service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:12.353744 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64881ba9-4958-40c0-8970-44ce5731316e service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:12.356730 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64881ba9-4958-40c0-8970-44ce5731316e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:12.384037 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64881ba9-4958-40c0-8970-44ce5731316e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:12.386456 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64881ba9-4958-40c0-8970-44ce5731316e service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:12.418820 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 660/1319] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:12 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:05:12.512266 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31974bcb-2db4-4858-8a2a-4be3a3e21d90 service cinder] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:12.524214 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31974bcb-2db4-4858-8a2a-4be3a3e21d90 service cinder] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:12.536253 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60de7b72-80a0-4ae1-8099-baae2ef95f22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:12.537142 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de7b72-80a0-4ae1-8099-baae2ef95f22 service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:12.537599 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de7b72-80a0-4ae1-8099-baae2ef95f22 service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:12.538047 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60de7b72-80a0-4ae1-8099-baae2ef95f22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:12.570066 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60de7b72-80a0-4ae1-8099-baae2ef95f22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:12.572029 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60de7b72-80a0-4ae1-8099-baae2ef95f22 service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:12.633211 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 660/1320] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:05:12.989967 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64881ba9-4958-40c0-8970-44ce5731316e service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:13.003447 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64881ba9-4958-40c0-8970-44ce5731316e service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:13.022070 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de657007-bac3-43a0-b79f-570506d4ab57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:13.022070 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de657007-bac3-43a0-b79f-570506d4ab57 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:13.022474 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de657007-bac3-43a0-b79f-570506d4ab57 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:13.022474 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de657007-bac3-43a0-b79f-570506d4ab57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:13.173228 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de657007-bac3-43a0-b79f-570506d4ab57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:13.174841 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de657007-bac3-43a0-b79f-570506d4ab57 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:13.198588 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 661/1321] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:12 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:05:16.072713 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60de7b72-80a0-4ae1-8099-baae2ef95f22 service cinder] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:16.118131 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:16.303748 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60de7b72-80a0-4ae1-8099-baae2ef95f22 service cinder] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:16.314998 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7504e-0f1f-4bfb-abc2-e92357d49668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:16.315960 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7504e-0f1f-4bfb-abc2-e92357d49668 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:16.316420 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7504e-0f1f-4bfb-abc2-e92357d49668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:16.316852 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7504e-0f1f-4bfb-abc2-e92357d49668 admin admin] PATH_INFO: `/v3/users/8b4d77a96af94f71b76c7b00390b56a3` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:16.323887 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b7504e-0f1f-4bfb-abc2-e92357d49668 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b4d77a96af94f71b76c7b00390b56a3)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:16.325230 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: WARNING py.warnings [None req-83b7504e-0f1f-4bfb-abc2-e92357d49668 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 20:05:16.325230 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: warnings.warn(msg) Oct 23 20:05:16.325230 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: Oct 23 20:05:16.327226 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b7504e-0f1f-4bfb-abc2-e92357d49668 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:16.531743 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-83b7504e-0f1f-4bfb-abc2-e92357d49668 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b4d77a96af94f71b76c7b00390b56a3'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:16.594673 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-83b7504e-0f1f-4bfb-abc2-e92357d49668 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b4d77a96af94f71b76c7b00390b56a3'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:16.619075 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-83b7504e-0f1f-4bfb-abc2-e92357d49668 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b4d77a96af94f71b76c7b00390b56a3'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:16.635381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-83b7504e-0f1f-4bfb-abc2-e92357d49668 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b4d77a96af94f71b76c7b00390b56a3'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:16.636265 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-83b7504e-0f1f-4bfb-abc2-e92357d49668 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b4d77a96af94f71b76c7b00390b56a3'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:16.652230 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 661/1322] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:05:16 2022] DELETE /identity/v3/users/8b4d77a96af94f71b76c7b00390b56a3 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:16.659472 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6f1da-0634-4063-b4c2-c9cc9f86be85 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:16.660749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6f1da-0634-4063-b4c2-c9cc9f86be85 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:16.661331 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6f1da-0634-4063-b4c2-c9cc9f86be85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:16.714861 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-4ce6f1da-0634-4063-b4c2-c9cc9f86be85 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:16.716953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4ce6f1da-0634-4063-b4c2-c9cc9f86be85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:16.896224 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 662/1323] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:17.122737 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7504e-0f1f-4bfb-abc2-e92357d49668 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:17.125089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-83b7504e-0f1f-4bfb-abc2-e92357d49668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:17.244000 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7504e-0f1f-4bfb-abc2-e92357d49668 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:17.255006 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf44c37-3fe8-4074-a839-461e52b54460 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:17.256689 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf44c37-3fe8-4074-a839-461e52b54460 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:17.257256 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf44c37-3fe8-4074-a839-461e52b54460 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:17.257991 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf44c37-3fe8-4074-a839-461e52b54460 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:17.278559 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf44c37-3fe8-4074-a839-461e52b54460 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:17.280308 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf44c37-3fe8-4074-a839-461e52b54460 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:17.298568 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 662/1324] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:05:17.597670 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200261aa-4929-4b44-beeb-21c2c14d41c2 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:17.597926 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200261aa-4929-4b44-beeb-21c2c14d41c2 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:17.598278 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200261aa-4929-4b44-beeb-21c2c14d41c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:17.669877 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-200261aa-4929-4b44-beeb-21c2c14d41c2 None None] MFA Rules not processed for user `a571b802668942e2b612ef125fe34ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:17.674651 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-200261aa-4929-4b44-beeb-21c2c14d41c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:17.770735 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:17.889131 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 663/1325] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:17.900849 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b61f99-7183-4437-b840-d10b26ff2158 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:17.901183 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b61f99-7183-4437-b840-d10b26ff2158 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:17.901421 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b61f99-7183-4437-b840-d10b26ff2158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:17.968540 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-e2b61f99-7183-4437-b840-d10b26ff2158 None None] MFA Rules not processed for user `a571b802668942e2b612ef125fe34ea6`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:17.971475 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-e2b61f99-7183-4437-b840-d10b26ff2158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:18.036586 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 663/1326] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:18.037783 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200261aa-4929-4b44-beeb-21c2c14d41c2 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:18.042124 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-200261aa-4929-4b44-beeb-21c2c14d41c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:18.245886 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200261aa-4929-4b44-beeb-21c2c14d41c2 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:18.261347 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdde56cc-6ef4-420f-906b-c9a1f296181d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:18.263005 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde56cc-6ef4-420f-906b-c9a1f296181d service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:18.263005 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde56cc-6ef4-420f-906b-c9a1f296181d service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:18.263005 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdde56cc-6ef4-420f-906b-c9a1f296181d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:18.277570 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdde56cc-6ef4-420f-906b-c9a1f296181d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:18.282125 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdde56cc-6ef4-420f-906b-c9a1f296181d service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:18.335345 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 664/1327] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:05:19.284294 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b14a4-1d21-47b8-ba34-16114c5feca9 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:19.285550 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b14a4-1d21-47b8-ba34-16114c5feca9 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:19.286004 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0b14a4-1d21-47b8-ba34-16114c5feca9 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:19.289262 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 664/1328] 10.222.0.13 () {54 vars in 912 bytes} [Sun Oct 23 20:05:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:05:19.297221 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e7a7c-af22-4e8c-b96b-3e4b7c3295c1 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:19.297688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e7a7c-af22-4e8c-b96b-3e4b7c3295c1 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:19.298046 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e7a7c-af22-4e8c-b96b-3e4b7c3295c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:19.387332 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-2a5e7a7c-af22-4e8c-b96b-3e4b7c3295c1 None None] MFA Rules not processed for user `ba8ff3413bc549d29486d1c724a1f383`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:19.389885 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2a5e7a7c-af22-4e8c-b96b-3e4b7c3295c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:19.413191 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 665/1329] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Oct 23 20:05:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:19.450753 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b14a4-1d21-47b8-ba34-16114c5feca9 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:19.471960 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0b14a4-1d21-47b8-ba34-16114c5feca9 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:19.482965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e7a7c-af22-4e8c-b96b-3e4b7c3295c1 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:19.484648 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2a5e7a7c-af22-4e8c-b96b-3e4b7c3295c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:19.485944 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591e8dcc-2587-472a-a530-a7cffca55b32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:19.488586 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e8dcc-2587-472a-a530-a7cffca55b32 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:19.494564 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e8dcc-2587-472a-a530-a7cffca55b32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:19.495065 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591e8dcc-2587-472a-a530-a7cffca55b32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:19.510996 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e7a7c-af22-4e8c-b96b-3e4b7c3295c1 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:19.520283 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591e8dcc-2587-472a-a530-a7cffca55b32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:19.523884 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c4276-13d9-489f-b27c-85625d2ac8a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:19.524645 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c4276-13d9-489f-b27c-85625d2ac8a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:19.525166 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c4276-13d9-489f-b27c-85625d2ac8a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:19.525448 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c4276-13d9-489f-b27c-85625d2ac8a5 admin admin] PATH_INFO: `/v3/projects/67e002b62fa7450b809a03d93496e7f1` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:19.527452 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591e8dcc-2587-472a-a530-a7cffca55b32 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:19.528330 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3c4276-13d9-489f-b27c-85625d2ac8a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=67e002b62fa7450b809a03d93496e7f1)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:19.529950 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: WARNING py.warnings [None req-8e3c4276-13d9-489f-b27c-85625d2ac8a5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 20:05:19.529950 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: warnings.warn(msg) Oct 23 20:05:19.529950 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: Oct 23 20:05:19.531095 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3c4276-13d9-489f-b27c-85625d2ac8a5 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:19.564558 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 665/1330] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:05:19.666982 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-8e3c4276-13d9-489f-b27c-85625d2ac8a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67e002b62fa7450b809a03d93496e7f1', 'request_id': 'req-8e3c4276-13d9-489f-b27c-85625d2ac8a5'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:19.681082 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-8e3c4276-13d9-489f-b27c-85625d2ac8a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67e002b62fa7450b809a03d93496e7f1', 'request_id': 'req-8e3c4276-13d9-489f-b27c-85625d2ac8a5'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:19.695907 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-8e3c4276-13d9-489f-b27c-85625d2ac8a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67e002b62fa7450b809a03d93496e7f1', 'request_id': 'req-8e3c4276-13d9-489f-b27c-85625d2ac8a5'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:19.734074 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-8e3c4276-13d9-489f-b27c-85625d2ac8a5 admin admin] The token cache is being invalidate because project 67e002b62fa7450b809a03d93496e7f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:05:19.734505 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-8e3c4276-13d9-489f-b27c-85625d2ac8a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:19.738676 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-8e3c4276-13d9-489f-b27c-85625d2ac8a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:19.740395 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 666/1331] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:05:19 2022] DELETE /identity/v3/projects/67e002b62fa7450b809a03d93496e7f1 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:05:19.746473 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2a597-1f0c-45dd-9877-69583bc0caf4 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:19.746762 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2a597-1f0c-45dd-9877-69583bc0caf4 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:19.748047 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e2a597-1f0c-45dd-9877-69583bc0caf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:19.795235 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-65e2a597-1f0c-45dd-9877-69583bc0caf4 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:19.798838 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-65e2a597-1f0c-45dd-9877-69583bc0caf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:19.832477 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:19.956545 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 666/1332] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:19.960335 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87203e40-b5f6-4edd-b4cc-b940932c91c5 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:19.961090 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87203e40-b5f6-4edd-b4cc-b940932c91c5 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:19.961577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87203e40-b5f6-4edd-b4cc-b940932c91c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:20.001901 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-87203e40-b5f6-4edd-b4cc-b940932c91c5 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:20.003415 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-87203e40-b5f6-4edd-b4cc-b940932c91c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:20.027949 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 667/1333] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:20.032112 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e2a597-1f0c-45dd-9877-69583bc0caf4 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:20.042980 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e2a597-1f0c-45dd-9877-69583bc0caf4 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:20.056903 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ff994-691d-4004-80ec-6026710b56d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:20.057684 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ff994-691d-4004-80ec-6026710b56d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:20.058443 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ff994-691d-4004-80ec-6026710b56d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:20.059434 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7ff994-691d-4004-80ec-6026710b56d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:20.060268 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7ff994-691d-4004-80ec-6026710b56d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:05:20.061037 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7ff994-691d-4004-80ec-6026710b56d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:20.069337 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7ff994-691d-4004-80ec-6026710b56d9 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:20.117879 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 667/1334] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:05:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:05:20.120033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87203e40-b5f6-4edd-b4cc-b940932c91c5 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:20.134926 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87203e40-b5f6-4edd-b4cc-b940932c91c5 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:20.142976 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd657b-a0e9-416f-bc03-db28998acde7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:20.143615 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd657b-a0e9-416f-bc03-db28998acde7 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:20.143901 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd657b-a0e9-416f-bc03-db28998acde7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:20.144181 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfd657b-a0e9-416f-bc03-db28998acde7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:20.144988 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd657b-a0e9-416f-bc03-db28998acde7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:20.146967 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfd657b-a0e9-416f-bc03-db28998acde7 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:20.194461 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:20.240159 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 668/1335] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:05:20 2022] POST /identity/v3/projects => generated 373 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:05:20.240382 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ff994-691d-4004-80ec-6026710b56d9 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:20.253558 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7ff994-691d-4004-80ec-6026710b56d9 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:20.354506 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b217602-a460-40ae-aa28-a02e750e4091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:20.356015 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b217602-a460-40ae-aa28-a02e750e4091 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:20.356699 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b217602-a460-40ae-aa28-a02e750e4091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:20.357135 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b217602-a460-40ae-aa28-a02e750e4091 admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:20.358613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b217602-a460-40ae-aa28-a02e750e4091 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:20.362167 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b217602-a460-40ae-aa28-a02e750e4091 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:20.396422 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 668/1336] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:05:20 2022] POST /identity/v3/users => generated 287 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:20.400616 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd657b-a0e9-416f-bc03-db28998acde7 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:20.415943 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfd657b-a0e9-416f-bc03-db28998acde7 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:20.421513 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c64c7f-47e6-4393-80f0-0157988f171c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:20.422656 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c64c7f-47e6-4393-80f0-0157988f171c admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:20.423169 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c64c7f-47e6-4393-80f0-0157988f171c admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:20.423635 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c64c7f-47e6-4393-80f0-0157988f171c admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:20.424978 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c64c7f-47e6-4393-80f0-0157988f171c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:20.427130 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c64c7f-47e6-4393-80f0-0157988f171c admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:20.449296 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b217602-a460-40ae-aa28-a02e750e4091 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:20.463310 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b217602-a460-40ae-aa28-a02e750e4091 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:20.464883 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 669/1337] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:20.479111 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f24f71-5aea-4d9c-9ba0-42336efa15b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:20.479852 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f24f71-5aea-4d9c-9ba0-42336efa15b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:20.480253 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f24f71-5aea-4d9c-9ba0-42336efa15b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:20.480647 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f24f71-5aea-4d9c-9ba0-42336efa15b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:20.481608 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f24f71-5aea-4d9c-9ba0-42336efa15b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:20.483407 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f24f71-5aea-4d9c-9ba0-42336efa15b8 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:20.510629 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 669/1338] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:20 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 20:05:20.511857 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c64c7f-47e6-4393-80f0-0157988f171c admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:20.532860 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c64c7f-47e6-4393-80f0-0157988f171c admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:20.549247 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8021507-4f00-49e2-b397-1a55a39789a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:20.549965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8021507-4f00-49e2-b397-1a55a39789a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:20.550254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8021507-4f00-49e2-b397-1a55a39789a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:20.550565 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8021507-4f00-49e2-b397-1a55a39789a6 admin admin] PATH_INFO: `/v3/projects/31293ae5a4de4ae3a39833ec75e4cfd4/users/dc5abbe6837d49f692fa3a76cb275388/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:20.579320 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8021507-4f00-49e2-b397-1a55a39789a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31293ae5a4de4ae3a39833ec75e4cfd4, user_id=dc5abbe6837d49f692fa3a76cb275388, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:20.582602 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8021507-4f00-49e2-b397-1a55a39789a6 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:20.625478 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 670/1339] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:20 2022] PUT /identity/v3/projects/31293ae5a4de4ae3a39833ec75e4cfd4/users/dc5abbe6837d49f692fa3a76cb275388/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:20.639565 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c0347-04e4-4bf4-a3e0-85434326a5d0 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:20.640254 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c0347-04e4-4bf4-a3e0-85434326a5d0 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:20.640791 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c0347-04e4-4bf4-a3e0-85434326a5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:20.761787 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-349c0347-04e4-4bf4-a3e0-85434326a5d0 None None] MFA Rules not processed for user `dc5abbe6837d49f692fa3a76cb275388`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:20.763068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-349c0347-04e4-4bf4-a3e0-85434326a5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:20.804452 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8021507-4f00-49e2-b397-1a55a39789a6 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:20.806549 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e8021507-4f00-49e2-b397-1a55a39789a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:20.938467 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 670/1340] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:20.942793 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349c0347-04e4-4bf4-a3e0-85434326a5d0 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:20.985822 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8021507-4f00-49e2-b397-1a55a39789a6 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:21.004043 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b6eee-868f-4fa7-bf41-c54171c128c5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312489c49a304ca9aacebd9843a4b65f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:21.004817 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b6eee-868f-4fa7-bf41-c54171c128c5 service placement] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:21.005216 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b6eee-868f-4fa7-bf41-c54171c128c5 service placement] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:21.005609 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b6eee-868f-4fa7-bf41-c54171c128c5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:21.013653 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:21.016107 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5b7b6eee-868f-4fa7-bf41-c54171c128c5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:21.059398 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7b6eee-868f-4fa7-bf41-c54171c128c5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:21.062395 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7b6eee-868f-4fa7-bf41-c54171c128c5 service placement] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:21.091791 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 671/1341] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:05:21.161446 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349c0347-04e4-4bf4-a3e0-85434326a5d0 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:21.170369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e39ce58-8608-44d9-b25b-cb216a2b14be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:21.171154 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39ce58-8608-44d9-b25b-cb216a2b14be admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:21.171370 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39ce58-8608-44d9-b25b-cb216a2b14be admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:21.171610 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39ce58-8608-44d9-b25b-cb216a2b14be admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:21.172324 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e39ce58-8608-44d9-b25b-cb216a2b14be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:21.175203 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e39ce58-8608-44d9-b25b-cb216a2b14be admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:21.251892 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 671/1342] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:05:20 2022] POST /identity/v3/projects => generated 371 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:05:21.254301 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b6eee-868f-4fa7-bf41-c54171c128c5 service placement] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:21.269676 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7b6eee-868f-4fa7-bf41-c54171c128c5 service placement] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:21.407048 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a253344e-a58d-4c0c-b562-180f179e2388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:21.408392 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a253344e-a58d-4c0c-b562-180f179e2388 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:21.408909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a253344e-a58d-4c0c-b562-180f179e2388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:21.409884 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a253344e-a58d-4c0c-b562-180f179e2388 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:21.410827 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a253344e-a58d-4c0c-b562-180f179e2388 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:21.413278 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a253344e-a58d-4c0c-b562-180f179e2388 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:21.459491 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 672/1343] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:05:21 2022] POST /identity/v3/users => generated 292 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:05:21.463778 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e39ce58-8608-44d9-b25b-cb216a2b14be admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:21.479113 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e39ce58-8608-44d9-b25b-cb216a2b14be admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:21.514662 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093df00d-17c0-485a-ad06-693f3dd1cc59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:21.527123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093df00d-17c0-485a-ad06-693f3dd1cc59 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:21.527123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093df00d-17c0-485a-ad06-693f3dd1cc59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:21.527123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093df00d-17c0-485a-ad06-693f3dd1cc59 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:21.527123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093df00d-17c0-485a-ad06-693f3dd1cc59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:21.539983 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093df00d-17c0-485a-ad06-693f3dd1cc59 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:21.560829 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 672/1344] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:21.563974 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a253344e-a58d-4c0c-b562-180f179e2388 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:21.593388 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a253344e-a58d-4c0c-b562-180f179e2388 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:21.631172 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca3fa8-4f4f-44e7-a8b4-873be5424903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:21.633184 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca3fa8-4f4f-44e7-a8b4-873be5424903 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:21.633643 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca3fa8-4f4f-44e7-a8b4-873be5424903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:21.634094 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca3fa8-4f4f-44e7-a8b4-873be5424903 admin admin] PATH_INFO: `/v3/projects/f78dcbb362d743138dfa1f4f71550b77/users/db2b4eb59bfc4523a91a53f53e82547c/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:21.667246 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca3fa8-4f4f-44e7-a8b4-873be5424903 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f78dcbb362d743138dfa1f4f71550b77, user_id=db2b4eb59bfc4523a91a53f53e82547c, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:21.675009 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca3fa8-4f4f-44e7-a8b4-873be5424903 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:21.714501 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 673/1345] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:21 2022] PUT /identity/v3/projects/f78dcbb362d743138dfa1f4f71550b77/users/db2b4eb59bfc4523a91a53f53e82547c/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:21.718353 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093df00d-17c0-485a-ad06-693f3dd1cc59 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:21.736652 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093df00d-17c0-485a-ad06-693f3dd1cc59 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:22.025412 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e86d9e-4c83-4f61-af88-8349519c8deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:22.026058 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e86d9e-4c83-4f61-af88-8349519c8deb admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:22.026375 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e86d9e-4c83-4f61-af88-8349519c8deb admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:22.026657 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e86d9e-4c83-4f61-af88-8349519c8deb admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:22.027323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e86d9e-4c83-4f61-af88-8349519c8deb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:22.031107 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e86d9e-4c83-4f61-af88-8349519c8deb admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:22.078149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 673/1346] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:21 2022] GET /identity/v3/roles => generated 1420 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:22.079334 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca3fa8-4f4f-44e7-a8b4-873be5424903 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:22.099070 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca3fa8-4f4f-44e7-a8b4-873be5424903 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:22.112608 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d325a-e59f-49f8-a14b-ae2914297411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:22.113314 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d325a-e59f-49f8-a14b-ae2914297411 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:22.113599 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d325a-e59f-49f8-a14b-ae2914297411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:22.113878 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00d325a-e59f-49f8-a14b-ae2914297411 admin admin] PATH_INFO: `/v3/projects/f78dcbb362d743138dfa1f4f71550b77/users/db2b4eb59bfc4523a91a53f53e82547c/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:22.122955 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d325a-e59f-49f8-a14b-ae2914297411 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f78dcbb362d743138dfa1f4f71550b77, user_id=db2b4eb59bfc4523a91a53f53e82547c, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:22.128074 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00d325a-e59f-49f8-a14b-ae2914297411 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:22.150840 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:22.192751 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 674/1347] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:22 2022] PUT /identity/v3/projects/f78dcbb362d743138dfa1f4f71550b77/users/db2b4eb59bfc4523a91a53f53e82547c/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:22.203511 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58bc8f-0c88-4bff-b5a4-1822ac11200b None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:22.203798 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58bc8f-0c88-4bff-b5a4-1822ac11200b None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:22.204080 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58bc8f-0c88-4bff-b5a4-1822ac11200b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:22.290265 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-6d58bc8f-0c88-4bff-b5a4-1822ac11200b None None] MFA Rules not processed for user `db2b4eb59bfc4523a91a53f53e82547c`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:22.292214 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-6d58bc8f-0c88-4bff-b5a4-1822ac11200b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:22.393766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d325a-e59f-49f8-a14b-ae2914297411 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:22.398760 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e00d325a-e59f-49f8-a14b-ae2914297411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:22.459181 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 674/1348] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:22 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:22.476093 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d58bc8f-0c88-4bff-b5a4-1822ac11200b None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:22.479415 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-6d58bc8f-0c88-4bff-b5a4-1822ac11200b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:22.509594 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:22.542707 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00d325a-e59f-49f8-a14b-ae2914297411 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:22.554722 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd5bf1b-88ec-4c0f-8ee9-b5576dc7c728 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:22.555384 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5bf1b-88ec-4c0f-8ee9-b5576dc7c728 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:22.555669 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5bf1b-88ec-4c0f-8ee9-b5576dc7c728 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:22.555945 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5bf1b-88ec-4c0f-8ee9-b5576dc7c728 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:22.560415 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-bbd5bf1b-88ec-4c0f-8ee9-b5576dc7c728 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:22.627652 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d58bc8f-0c88-4bff-b5a4-1822ac11200b None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:22.634305 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fc5c5e-0af9-48c2-a573-4d95e8b2c476 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:22.635275 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc5c5e-0af9-48c2-a573-4d95e8b2c476 service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:22.637010 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc5c5e-0af9-48c2-a573-4d95e8b2c476 service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:22.637421 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fc5c5e-0af9-48c2-a573-4d95e8b2c476 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:22.659372 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fc5c5e-0af9-48c2-a573-4d95e8b2c476 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:22.661071 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fc5c5e-0af9-48c2-a573-4d95e8b2c476 service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:22.679036 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 675/1349] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:22 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 20:05:22.695143 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d15228-6c74-496f-a24c-e5fda60d2634 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:22.700629 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d15228-6c74-496f-a24c-e5fda60d2634 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:22.701076 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d15228-6c74-496f-a24c-e5fda60d2634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:22.721843 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd5bf1b-88ec-4c0f-8ee9-b5576dc7c728 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:22.723374 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd5bf1b-88ec-4c0f-8ee9-b5576dc7c728 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:22.752814 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 675/1350] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:22 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:05:22.804480 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-13d15228-6c74-496f-a24c-e5fda60d2634 None None] MFA Rules not processed for user `7aca00ea597140c5be97a6a4fa33f9c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:22.805788 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-13d15228-6c74-496f-a24c-e5fda60d2634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:22.979789 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 676/1351] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:22.981891 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd5bf1b-88ec-4c0f-8ee9-b5576dc7c728 service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:22.983384 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-bbd5bf1b-88ec-4c0f-8ee9-b5576dc7c728 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:23.181073 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd5bf1b-88ec-4c0f-8ee9-b5576dc7c728 service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:23.200780 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd59d3b0-de4d-4fd6-a90e-2b7d18a6dd48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:23.201801 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59d3b0-de4d-4fd6-a90e-2b7d18a6dd48 service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:23.201801 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59d3b0-de4d-4fd6-a90e-2b7d18a6dd48 service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:23.202029 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd59d3b0-de4d-4fd6-a90e-2b7d18a6dd48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:23.253957 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd59d3b0-de4d-4fd6-a90e-2b7d18a6dd48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:23.256983 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd59d3b0-de4d-4fd6-a90e-2b7d18a6dd48 service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:23.287778 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 676/1352] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:05:23.930481 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d15228-6c74-496f-a24c-e5fda60d2634 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:24.061525 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d15228-6c74-496f-a24c-e5fda60d2634 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:24.067645 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c1da4-a73f-4c9f-830e-8cd7702616df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:24.068921 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c1da4-a73f-4c9f-830e-8cd7702616df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:24.069414 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c1da4-a73f-4c9f-830e-8cd7702616df admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:24.069965 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2c1da4-a73f-4c9f-830e-8cd7702616df admin admin] PATH_INFO: `/v3/users/dc5abbe6837d49f692fa3a76cb275388` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:24.072346 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c1da4-a73f-4c9f-830e-8cd7702616df admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc5abbe6837d49f692fa3a76cb275388)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:24.074991 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2c1da4-a73f-4c9f-830e-8cd7702616df admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:24.188351 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:24.190430 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-5c2c1da4-a73f-4c9f-830e-8cd7702616df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc5abbe6837d49f692fa3a76cb275388'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:24.231651 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-5c2c1da4-a73f-4c9f-830e-8cd7702616df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc5abbe6837d49f692fa3a76cb275388'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:24.244017 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-5c2c1da4-a73f-4c9f-830e-8cd7702616df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc5abbe6837d49f692fa3a76cb275388'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:24.292530 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-5c2c1da4-a73f-4c9f-830e-8cd7702616df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc5abbe6837d49f692fa3a76cb275388'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:24.293671 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-5c2c1da4-a73f-4c9f-830e-8cd7702616df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc5abbe6837d49f692fa3a76cb275388'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:24.305283 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 677/1353] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:05:23 2022] DELETE /identity/v3/users/dc5abbe6837d49f692fa3a76cb275388 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:24.309801 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3be63c-91ec-48c3-9672-db338b1dd0a7 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:24.310367 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3be63c-91ec-48c3-9672-db338b1dd0a7 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:24.310469 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3be63c-91ec-48c3-9672-db338b1dd0a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:24.353371 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-5e3be63c-91ec-48c3-9672-db338b1dd0a7 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:24.354919 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5e3be63c-91ec-48c3-9672-db338b1dd0a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:24.434841 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:24.499836 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 677/1354] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:24.507269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c1da4-a73f-4c9f-830e-8cd7702616df admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:24.509622 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-5c2c1da4-a73f-4c9f-830e-8cd7702616df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:24.648022 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2c1da4-a73f-4c9f-830e-8cd7702616df admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:24.652858 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee90d84-e866-4166-863c-a8ecba8a2b52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:24.653596 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee90d84-e866-4166-863c-a8ecba8a2b52 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:24.653818 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee90d84-e866-4166-863c-a8ecba8a2b52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:24.654093 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee90d84-e866-4166-863c-a8ecba8a2b52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:24.665812 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee90d84-e866-4166-863c-a8ecba8a2b52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:24.667657 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee90d84-e866-4166-863c-a8ecba8a2b52 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:24.680611 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 678/1355] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:05:24.955775 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60143ae0-0f3e-4d22-83ce-868af41ba51f None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:24.956055 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60143ae0-0f3e-4d22-83ce-868af41ba51f None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:24.956330 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60143ae0-0f3e-4d22-83ce-868af41ba51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:25.034078 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-60143ae0-0f3e-4d22-83ce-868af41ba51f None None] MFA Rules not processed for user `7cef68ee160348dea427e6c46b775e2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:25.040924 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-60143ae0-0f3e-4d22-83ce-868af41ba51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:25.190833 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 678/1356] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:24 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:25.206415 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee90d84-e866-4166-863c-a8ecba8a2b52 service neutron] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:25.208106 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-0ee90d84-e866-4166-863c-a8ecba8a2b52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:25.358834 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee90d84-e866-4166-863c-a8ecba8a2b52 service neutron] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:25.368091 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4e7d1-bce2-41ce-b683-64001e468687 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:25.369237 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4e7d1-bce2-41ce-b683-64001e468687 service nova] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:25.370681 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4e7d1-bce2-41ce-b683-64001e468687 service nova] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:25.371118 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4e7d1-bce2-41ce-b683-64001e468687 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:25.391222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4e7d1-bce2-41ce-b683-64001e468687 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:25.393354 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4e7d1-bce2-41ce-b683-64001e468687 service nova] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:25.407143 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 679/1357] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:25 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:05:25.640890 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60143ae0-0f3e-4d22-83ce-868af41ba51f None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:25.644016 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-60143ae0-0f3e-4d22-83ce-868af41ba51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:25.705938 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60143ae0-0f3e-4d22-83ce-868af41ba51f None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:25.728607 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f5c28-d87e-47ee-81f3-d9d82624e204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:25.729270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f5c28-d87e-47ee-81f3-d9d82624e204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:25.729551 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f5c28-d87e-47ee-81f3-d9d82624e204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:25.730021 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f5c28-d87e-47ee-81f3-d9d82624e204 admin admin] PATH_INFO: `/v3/projects/31293ae5a4de4ae3a39833ec75e4cfd4` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:25.731386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2f5c28-d87e-47ee-81f3-d9d82624e204 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31293ae5a4de4ae3a39833ec75e4cfd4)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:25.733332 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2f5c28-d87e-47ee-81f3-d9d82624e204 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:25.840875 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-fa2f5c28-d87e-47ee-81f3-d9d82624e204 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31293ae5a4de4ae3a39833ec75e4cfd4', 'request_id': 'req-fa2f5c28-d87e-47ee-81f3-d9d82624e204'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:25.854196 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-fa2f5c28-d87e-47ee-81f3-d9d82624e204 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31293ae5a4de4ae3a39833ec75e4cfd4', 'request_id': 'req-fa2f5c28-d87e-47ee-81f3-d9d82624e204'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:25.867074 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-fa2f5c28-d87e-47ee-81f3-d9d82624e204 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31293ae5a4de4ae3a39833ec75e4cfd4', 'request_id': 'req-fa2f5c28-d87e-47ee-81f3-d9d82624e204'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:25.899729 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-fa2f5c28-d87e-47ee-81f3-d9d82624e204 admin admin] The token cache is being invalidate because project 31293ae5a4de4ae3a39833ec75e4cfd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:05:25.900228 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-fa2f5c28-d87e-47ee-81f3-d9d82624e204 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:25.901803 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-fa2f5c28-d87e-47ee-81f3-d9d82624e204 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:25.903970 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 679/1358] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:05:25 2022] DELETE /identity/v3/projects/31293ae5a4de4ae3a39833ec75e4cfd4 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:25.908413 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4e7d1-bce2-41ce-b683-64001e468687 service nova] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:25.925760 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4e7d1-bce2-41ce-b683-64001e468687 service nova] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:26.029497 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:26.052588 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ca5143-9665-4730-8d78-9fdfd924348f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:26.053544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca5143-9665-4730-8d78-9fdfd924348f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:26.053921 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca5143-9665-4730-8d78-9fdfd924348f admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:26.054527 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ca5143-9665-4730-8d78-9fdfd924348f admin admin] PATH_INFO: `/v3/users/db2b4eb59bfc4523a91a53f53e82547c` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:26.056855 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ca5143-9665-4730-8d78-9fdfd924348f admin admin] RBAC: Authorizing `identity:delete_user(user_id=db2b4eb59bfc4523a91a53f53e82547c)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:26.059142 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ca5143-9665-4730-8d78-9fdfd924348f admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:26.219959 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-a6ca5143-9665-4730-8d78-9fdfd924348f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db2b4eb59bfc4523a91a53f53e82547c'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:26.282926 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-a6ca5143-9665-4730-8d78-9fdfd924348f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db2b4eb59bfc4523a91a53f53e82547c'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:26.301627 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-a6ca5143-9665-4730-8d78-9fdfd924348f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db2b4eb59bfc4523a91a53f53e82547c'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:26.318562 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-a6ca5143-9665-4730-8d78-9fdfd924348f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db2b4eb59bfc4523a91a53f53e82547c'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:26.320575 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-a6ca5143-9665-4730-8d78-9fdfd924348f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db2b4eb59bfc4523a91a53f53e82547c'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:26.333759 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 680/1359] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:05:25 2022] DELETE /identity/v3/users/db2b4eb59bfc4523a91a53f53e82547c => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:26.385818 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b47aae-bf49-4a90-baf8-f3a5ad7d44d9 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:26.386026 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b47aae-bf49-4a90-baf8-f3a5ad7d44d9 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:26.386336 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b47aae-bf49-4a90-baf8-f3a5ad7d44d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:26.451293 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-e1b47aae-bf49-4a90-baf8-f3a5ad7d44d9 None None] MFA Rules not processed for user `4b6a101d23df4b62ba7e0c35d28d2a3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:26.457318 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e1b47aae-bf49-4a90-baf8-f3a5ad7d44d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:26.560829 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:26.654975 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 680/1360] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:26 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:26.655796 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d371e6-4057-42a7-86cb-0541720fac05 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:26.656082 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d371e6-4057-42a7-86cb-0541720fac05 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:26.656357 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d371e6-4057-42a7-86cb-0541720fac05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:26.707482 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-40d371e6-4057-42a7-86cb-0541720fac05 None None] MFA Rules not processed for user `4b6a101d23df4b62ba7e0c35d28d2a3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:26.715643 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-40d371e6-4057-42a7-86cb-0541720fac05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:26.747958 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 681/1361] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:26 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:26.756252 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b47aae-bf49-4a90-baf8-f3a5ad7d44d9 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:26.766504 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e1b47aae-bf49-4a90-baf8-f3a5ad7d44d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:26.940325 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b47aae-bf49-4a90-baf8-f3a5ad7d44d9 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:26.948483 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1512497a-7c09-45f4-abcd-124833aee061 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:26.949392 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512497a-7c09-45f4-abcd-124833aee061 service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:26.949829 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512497a-7c09-45f4-abcd-124833aee061 service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:26.950455 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1512497a-7c09-45f4-abcd-124833aee061 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:26.982929 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1512497a-7c09-45f4-abcd-124833aee061 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:26.987341 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1512497a-7c09-45f4-abcd-124833aee061 service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:27.005762 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 681/1362] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:26 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:05:27.515425 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d371e6-4057-42a7-86cb-0541720fac05 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:27.518066 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-40d371e6-4057-42a7-86cb-0541720fac05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:27.701887 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d371e6-4057-42a7-86cb-0541720fac05 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:27.729345 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258430f2-5561-45ca-b642-f499486836de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:27.730184 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258430f2-5561-45ca-b642-f499486836de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:27.733136 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258430f2-5561-45ca-b642-f499486836de admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:27.733424 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258430f2-5561-45ca-b642-f499486836de admin admin] PATH_INFO: `/v3/projects/f78dcbb362d743138dfa1f4f71550b77` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:27.742620 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258430f2-5561-45ca-b642-f499486836de admin admin] RBAC: Authorizing `identity:delete_project(project_id=f78dcbb362d743138dfa1f4f71550b77)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:27.744591 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258430f2-5561-45ca-b642-f499486836de admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:27.781734 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a54e77-7696-4089-bc89-7ef554fad6f6 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:27.782171 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a54e77-7696-4089-bc89-7ef554fad6f6 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:27.782339 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a54e77-7696-4089-bc89-7ef554fad6f6 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:27.784265 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 682/1363] 10.222.0.13 () {54 vars in 912 bytes} [Sun Oct 23 20:05:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:05:27.790775 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4d9fb-8f01-4f1a-b871-b38016dc0b23 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:27.791113 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4d9fb-8f01-4f1a-b871-b38016dc0b23 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:27.792747 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c4d9fb-8f01-4f1a-b871-b38016dc0b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:27.861624 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-52c4d9fb-8f01-4f1a-b871-b38016dc0b23 None None] MFA Rules not processed for user `ba8ff3413bc549d29486d1c724a1f383`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:27.863300 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-52c4d9fb-8f01-4f1a-b871-b38016dc0b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:27.931507 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-258430f2-5561-45ca-b642-f499486836de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f78dcbb362d743138dfa1f4f71550b77', 'request_id': 'req-258430f2-5561-45ca-b642-f499486836de'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:27.958404 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-258430f2-5561-45ca-b642-f499486836de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f78dcbb362d743138dfa1f4f71550b77', 'request_id': 'req-258430f2-5561-45ca-b642-f499486836de'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:27.966858 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-258430f2-5561-45ca-b642-f499486836de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f78dcbb362d743138dfa1f4f71550b77', 'request_id': 'req-258430f2-5561-45ca-b642-f499486836de'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:27.968336 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:27.993966 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-258430f2-5561-45ca-b642-f499486836de admin admin] The token cache is being invalidate because project f78dcbb362d743138dfa1f4f71550b77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:05:27.994574 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-258430f2-5561-45ca-b642-f499486836de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:28.000040 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-258430f2-5561-45ca-b642-f499486836de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:28.005659 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 682/1364] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:05:27 2022] DELETE /identity/v3/projects/f78dcbb362d743138dfa1f4f71550b77 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:28.007852 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0b7ae-8122-4bc0-b5b6-18563f075a92 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:28.008189 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0b7ae-8122-4bc0-b5b6-18563f075a92 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:28.008565 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0b7ae-8122-4bc0-b5b6-18563f075a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:28.011287 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 683/1365] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Oct 23 20:05:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:05:28.038237 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4d9fb-8f01-4f1a-b871-b38016dc0b23 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:28.041079 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-52c4d9fb-8f01-4f1a-b871-b38016dc0b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:28.059822 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-24b0b7ae-8122-4bc0-b5b6-18563f075a92 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:28.061428 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c4d9fb-8f01-4f1a-b871-b38016dc0b23 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:28.062332 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-24b0b7ae-8122-4bc0-b5b6-18563f075a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:28.164208 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357279aa-2d45-423c-a3b4-57570bdf21ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:28.167545 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357279aa-2d45-423c-a3b4-57570bdf21ce service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:28.167851 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357279aa-2d45-423c-a3b4-57570bdf21ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:28.170380 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357279aa-2d45-423c-a3b4-57570bdf21ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:28.176672 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-357279aa-2d45-423c-a3b4-57570bdf21ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:28.215199 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357279aa-2d45-423c-a3b4-57570bdf21ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:28.220320 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357279aa-2d45-423c-a3b4-57570bdf21ce service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:28.238310 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 683/1366] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:28.245881 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586f033-2fa3-4d54-b8f6-f590559cbd9b None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:28.246324 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586f033-2fa3-4d54-b8f6-f590559cbd9b None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:28.246704 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586f033-2fa3-4d54-b8f6-f590559cbd9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:28.255902 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 684/1367] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:05:28.287359 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-e586f033-2fa3-4d54-b8f6-f590559cbd9b None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:28.289190 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-e586f033-2fa3-4d54-b8f6-f590559cbd9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:28.312905 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 684/1368] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:05:28.313987 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357279aa-2d45-423c-a3b4-57570bdf21ce service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:28.332614 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357279aa-2d45-423c-a3b4-57570bdf21ce service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:28.351587 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082713a1-ff77-4eb6-bfeb-810c8eee7681 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:28.352237 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082713a1-ff77-4eb6-bfeb-810c8eee7681 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:28.355988 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082713a1-ff77-4eb6-bfeb-810c8eee7681 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:28.356291 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082713a1-ff77-4eb6-bfeb-810c8eee7681 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:28.358749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082713a1-ff77-4eb6-bfeb-810c8eee7681 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:05:28.360806 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082713a1-ff77-4eb6-bfeb-810c8eee7681 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:28.367956 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082713a1-ff77-4eb6-bfeb-810c8eee7681 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:28.393710 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d33d32-0ea4-4644-864a-1a26fd519538 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:28.394119 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d33d32-0ea4-4644-864a-1a26fd519538 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:28.394523 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d33d32-0ea4-4644-864a-1a26fd519538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:28.405238 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:28.417212 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 685/1369] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:05:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:05:28.420141 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082713a1-ff77-4eb6-bfeb-810c8eee7681 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:28.431435 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-f6d33d32-0ea4-4644-864a-1a26fd519538 None None] MFA Rules not processed for user `7aca00ea597140c5be97a6a4fa33f9c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:28.444982 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-f6d33d32-0ea4-4644-864a-1a26fd519538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:28.453058 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082713a1-ff77-4eb6-bfeb-810c8eee7681 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:28.472062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9226c998-e63d-4646-b376-ef121a94dfb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:28.472712 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9226c998-e63d-4646-b376-ef121a94dfb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:28.473069 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9226c998-e63d-4646-b376-ef121a94dfb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:28.473348 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9226c998-e63d-4646-b376-ef121a94dfb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:28.478572 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9226c998-e63d-4646-b376-ef121a94dfb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:28.480615 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9226c998-e63d-4646-b376-ef121a94dfb9 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:28.554807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 686/1370] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:05:28 2022] POST /identity/v3/projects => generated 355 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:28.558903 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9226c998-e63d-4646-b376-ef121a94dfb9 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:28.582100 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 685/1371] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:05:28.642863 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9226c998-e63d-4646-b376-ef121a94dfb9 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:28.653452 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd72082-7299-4290-af7f-5d999cdcee49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:28.658982 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd72082-7299-4290-af7f-5d999cdcee49 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:28.659291 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd72082-7299-4290-af7f-5d999cdcee49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:28.659578 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd72082-7299-4290-af7f-5d999cdcee49 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:28.660263 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd72082-7299-4290-af7f-5d999cdcee49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:28.662593 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd72082-7299-4290-af7f-5d999cdcee49 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:28.697218 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 687/1372] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:05:28 2022] POST /identity/v3/users => generated 278 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:05:28.700470 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d33d32-0ea4-4644-864a-1a26fd519538 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:28.719046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d33d32-0ea4-4644-864a-1a26fd519538 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:28.725153 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c791d1ab-98dd-4be2-8875-072546e5fbd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:28.725887 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791d1ab-98dd-4be2-8875-072546e5fbd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:28.726373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791d1ab-98dd-4be2-8875-072546e5fbd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:28.726709 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c791d1ab-98dd-4be2-8875-072546e5fbd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:28.727441 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c791d1ab-98dd-4be2-8875-072546e5fbd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:28.729923 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c791d1ab-98dd-4be2-8875-072546e5fbd7 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:28.746266 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 686/1373] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:28 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:28.750921 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd72082-7299-4290-af7f-5d999cdcee49 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:28.782484 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd72082-7299-4290-af7f-5d999cdcee49 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:28.794082 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8151133a-9c6e-4ed9-b24b-fcd462ccf27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:28.795748 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151133a-9c6e-4ed9-b24b-fcd462ccf27b admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:28.797358 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151133a-9c6e-4ed9-b24b-fcd462ccf27b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:28.798177 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8151133a-9c6e-4ed9-b24b-fcd462ccf27b admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:28.799602 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8151133a-9c6e-4ed9-b24b-fcd462ccf27b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:28.802326 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8151133a-9c6e-4ed9-b24b-fcd462ccf27b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:28.819945 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 688/1374] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:28 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:28.823710 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c791d1ab-98dd-4be2-8875-072546e5fbd7 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:28.838557 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c791d1ab-98dd-4be2-8875-072546e5fbd7 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:28.850055 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcc0f7a-cc89-458f-97d7-05024eb7d822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:28.852053 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc0f7a-cc89-458f-97d7-05024eb7d822 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:28.852492 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc0f7a-cc89-458f-97d7-05024eb7d822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:28.852906 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcc0f7a-cc89-458f-97d7-05024eb7d822 admin admin] PATH_INFO: `/v3/projects/ef0abe2425994dcab2d2e2368a0de717/users/6e10299fabad4dc6895a65631025584b/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:28.891023 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc0f7a-cc89-458f-97d7-05024eb7d822 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef0abe2425994dcab2d2e2368a0de717, user_id=6e10299fabad4dc6895a65631025584b, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:28.900013 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcc0f7a-cc89-458f-97d7-05024eb7d822 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:28.930653 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 687/1375] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:28 2022] PUT /identity/v3/projects/ef0abe2425994dcab2d2e2368a0de717/users/6e10299fabad4dc6895a65631025584b/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:28.949609 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1a173-71ec-4c40-af89-c982ceb895f2 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:28.949976 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1a173-71ec-4c40-af89-c982ceb895f2 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:28.950364 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1a173-71ec-4c40-af89-c982ceb895f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:29.075040 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-46b1a173-71ec-4c40-af89-c982ceb895f2 None None] MFA Rules not processed for user `6e10299fabad4dc6895a65631025584b`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:29.076680 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-46b1a173-71ec-4c40-af89-c982ceb895f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:29.227176 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b4a5f-1e29-445c-81bb-6f7c44be9572 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:29.227484 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b4a5f-1e29-445c-81bb-6f7c44be9572 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:29.227762 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7b4a5f-1e29-445c-81bb-6f7c44be9572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:29.305048 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 689/1376] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:28 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 356 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:29.310455 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1a173-71ec-4c40-af89-c982ceb895f2 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:29.349544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-cf7b4a5f-1e29-445c-81bb-6f7c44be9572 None None] MFA Rules not processed for user `4b6a101d23df4b62ba7e0c35d28d2a3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:29.352112 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-cf7b4a5f-1e29-445c-81bb-6f7c44be9572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:29.428064 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:29.466211 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b1a173-71ec-4c40-af89-c982ceb895f2 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:29.475628 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cfabbb-c528-438e-8332-dcce5dd17f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:29.476689 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cfabbb-c528-438e-8332-dcce5dd17f4c admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:29.477165 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cfabbb-c528-438e-8332-dcce5dd17f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:29.477824 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cfabbb-c528-438e-8332-dcce5dd17f4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:29.478554 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cfabbb-c528-438e-8332-dcce5dd17f4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:29.489716 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cfabbb-c528-438e-8332-dcce5dd17f4c admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:29.506388 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 688/1377] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:29 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:29.567864 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 690/1378] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:05:29 2022] POST /identity/v3/projects => generated 355 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:29.568470 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b4a5f-1e29-445c-81bb-6f7c44be9572 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:29.574993 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:29.588402 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7b4a5f-1e29-445c-81bb-6f7c44be9572 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:29.795555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6315d04-14b8-49e8-86d6-ca4b7644d27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:29.796453 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6315d04-14b8-49e8-86d6-ca4b7644d27c admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:29.796838 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6315d04-14b8-49e8-86d6-ca4b7644d27c admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:29.797220 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6315d04-14b8-49e8-86d6-ca4b7644d27c admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:29.798049 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6315d04-14b8-49e8-86d6-ca4b7644d27c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:29.800202 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6315d04-14b8-49e8-86d6-ca4b7644d27c admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:29.829607 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cfabbb-c528-438e-8332-dcce5dd17f4c admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:29.831645 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 689/1379] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:05:29 2022] POST /identity/v3/users => generated 284 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:05:29.840953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cfabbb-c528-438e-8332-dcce5dd17f4c admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:29.856747 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c75970-2eda-4b6d-b4d5-6be3d10ce010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:29.857390 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c75970-2eda-4b6d-b4d5-6be3d10ce010 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:29.857670 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c75970-2eda-4b6d-b4d5-6be3d10ce010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:29.857941 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c75970-2eda-4b6d-b4d5-6be3d10ce010 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:29.858805 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c75970-2eda-4b6d-b4d5-6be3d10ce010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:29.860396 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c75970-2eda-4b6d-b4d5-6be3d10ce010 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:29.890742 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 691/1380] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:29.894779 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6315d04-14b8-49e8-86d6-ca4b7644d27c admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:29.905726 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6315d04-14b8-49e8-86d6-ca4b7644d27c admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:29.913064 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c5a9d-8a74-4d52-833f-91b80ffe9666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:29.913819 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c5a9d-8a74-4d52-833f-91b80ffe9666 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:29.914096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c5a9d-8a74-4d52-833f-91b80ffe9666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:29.914401 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c5a9d-8a74-4d52-833f-91b80ffe9666 admin admin] PATH_INFO: `/v3/projects/05889dc8180c49eba24edc7510cd9714/users/1b08c2d95a8e447490d4f94db18d3c3d/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:29.949872 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c5a9d-8a74-4d52-833f-91b80ffe9666 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05889dc8180c49eba24edc7510cd9714, user_id=1b08c2d95a8e447490d4f94db18d3c3d, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:29.952783 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c5a9d-8a74-4d52-833f-91b80ffe9666 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:29.969741 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 690/1381] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:29 2022] PUT /identity/v3/projects/05889dc8180c49eba24edc7510cd9714/users/1b08c2d95a8e447490d4f94db18d3c3d/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:29.973854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c75970-2eda-4b6d-b4d5-6be3d10ce010 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:29.988444 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c75970-2eda-4b6d-b4d5-6be3d10ce010 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:30.136188 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11c469-21e5-42bf-b50b-cf0b3d0cfa66 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:30.136921 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11c469-21e5-42bf-b50b-cf0b3d0cfa66 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:30.136921 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11c469-21e5-42bf-b50b-cf0b3d0cfa66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:30.187742 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-6b11c469-21e5-42bf-b50b-cf0b3d0cfa66 None None] MFA Rules not processed for user `7cef68ee160348dea427e6c46b775e2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:30.192318 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-6b11c469-21e5-42bf-b50b-cf0b3d0cfa66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:30.201865 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d3aa87-d0f8-4a6d-86d9-b305f7356cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:30.202773 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3aa87-d0f8-4a6d-86d9-b305f7356cd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:30.203187 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3aa87-d0f8-4a6d-86d9-b305f7356cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:30.203593 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d3aa87-d0f8-4a6d-86d9-b305f7356cd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:30.207827 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d3aa87-d0f8-4a6d-86d9-b305f7356cd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:30.214496 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d3aa87-d0f8-4a6d-86d9-b305f7356cd0 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:30.245176 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 692/1382] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:29 2022] GET /identity/v3/roles => generated 1420 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:30.247055 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d3aa87-d0f8-4a6d-86d9-b305f7356cd0 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:30.273658 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d3aa87-d0f8-4a6d-86d9-b305f7356cd0 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:30.309464 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d6cc0-a84d-4c16-9c83-4d5ffafe4d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:30.310466 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d6cc0-a84d-4c16-9c83-4d5ffafe4d78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:30.310749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d6cc0-a84d-4c16-9c83-4d5ffafe4d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:30.315120 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0d6cc0-a84d-4c16-9c83-4d5ffafe4d78 admin admin] PATH_INFO: `/v3/projects/05889dc8180c49eba24edc7510cd9714/users/1b08c2d95a8e447490d4f94db18d3c3d/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:30.320582 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d6cc0-a84d-4c16-9c83-4d5ffafe4d78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05889dc8180c49eba24edc7510cd9714, user_id=1b08c2d95a8e447490d4f94db18d3c3d, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:30.324042 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0d6cc0-a84d-4c16-9c83-4d5ffafe4d78 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:30.363545 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 691/1383] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:30 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:05:30.367922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 693/1384] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:30 2022] PUT /identity/v3/projects/05889dc8180c49eba24edc7510cd9714/users/1b08c2d95a8e447490d4f94db18d3c3d/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:30.384279 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215f8f9-9ca8-4b2a-903f-ac042fda1cbf None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:30.384583 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215f8f9-9ca8-4b2a-903f-ac042fda1cbf None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:30.384865 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9215f8f9-9ca8-4b2a-903f-ac042fda1cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:30.448075 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-9215f8f9-9ca8-4b2a-903f-ac042fda1cbf None None] MFA Rules not processed for user `1b08c2d95a8e447490d4f94db18d3c3d`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:30.449326 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-9215f8f9-9ca8-4b2a-903f-ac042fda1cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:30.678303 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:30.691852 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 692/1385] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:30 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:30.704387 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d6cc0-a84d-4c16-9c83-4d5ffafe4d78 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:30.707662 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-fc0d6cc0-a84d-4c16-9c83-4d5ffafe4d78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:30.876671 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0d6cc0-a84d-4c16-9c83-4d5ffafe4d78 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:30.890207 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a23b4-1232-4c78-b0d1-7d4869d89bb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:30.901878 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a23b4-1232-4c78-b0d1-7d4869d89bb7 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:30.902140 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a23b4-1232-4c78-b0d1-7d4869d89bb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:30.903956 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4a23b4-1232-4c78-b0d1-7d4869d89bb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:30.932500 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4a23b4-1232-4c78-b0d1-7d4869d89bb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:30.936490 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4a23b4-1232-4c78-b0d1-7d4869d89bb7 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:30.937783 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5495b8e-6871-433f-b8b8-babfee4121de None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:30.938076 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5495b8e-6871-433f-b8b8-babfee4121de None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:30.938394 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5495b8e-6871-433f-b8b8-babfee4121de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:30.960677 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 694/1386] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:30 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:05:31.059944 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-c5495b8e-6871-433f-b8b8-babfee4121de None None] MFA Rules not processed for user `ff5cfe4d31614e2db9059c69116297cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:31.061560 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-c5495b8e-6871-433f-b8b8-babfee4121de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:31.171683 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a23b4-1232-4c78-b0d1-7d4869d89bb7 service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:31.182844 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4a23b4-1232-4c78-b0d1-7d4869d89bb7 service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:31.197006 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db0a488-4633-4617-9e93-55fce72ba4d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:31.208834 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0a488-4633-4617-9e93-55fce72ba4d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:31.210228 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0a488-4633-4617-9e93-55fce72ba4d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:31.210644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db0a488-4633-4617-9e93-55fce72ba4d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:31.250631 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 693/1387] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Oct 23 20:05:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:05:31.358408 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:31.387210 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db0a488-4633-4617-9e93-55fce72ba4d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:31.389012 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db0a488-4633-4617-9e93-55fce72ba4d1 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:31.410993 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 695/1388] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:31 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:05:31.571450 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5495b8e-6871-433f-b8b8-babfee4121de None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:31.576222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-c5495b8e-6871-433f-b8b8-babfee4121de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:31.968232 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5495b8e-6871-433f-b8b8-babfee4121de None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:31.981542 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50de45bb-ee25-4b0d-b54a-cdf010173939 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15f603d1e7f54751aa79317e2ea00a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:31.982393 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de45bb-ee25-4b0d-b54a-cdf010173939 service swift] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:31.982806 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de45bb-ee25-4b0d-b54a-cdf010173939 service swift] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:31.983246 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50de45bb-ee25-4b0d-b54a-cdf010173939 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:32.036586 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db0a488-4633-4617-9e93-55fce72ba4d1 service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:32.039075 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7db0a488-4633-4617-9e93-55fce72ba4d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:32.047830 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50de45bb-ee25-4b0d-b54a-cdf010173939 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:32.059358 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50de45bb-ee25-4b0d-b54a-cdf010173939 service swift] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:32.109983 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 694/1389] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Oct 23 20:05:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 20:05:32.312809 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db0a488-4633-4617-9e93-55fce72ba4d1 service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:32.323354 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84239728-ee5e-4206-9b05-213378058be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:32.324514 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84239728-ee5e-4206-9b05-213378058be6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:32.325185 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84239728-ee5e-4206-9b05-213378058be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:32.325682 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84239728-ee5e-4206-9b05-213378058be6 admin admin] PATH_INFO: `/v3/users/9a3030e91531434f9bba4bb518ebf424` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:32.327838 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84239728-ee5e-4206-9b05-213378058be6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a3030e91531434f9bba4bb518ebf424)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:32.330304 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84239728-ee5e-4206-9b05-213378058be6 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:32.466167 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-84239728-ee5e-4206-9b05-213378058be6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a3030e91531434f9bba4bb518ebf424'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:32.557434 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-84239728-ee5e-4206-9b05-213378058be6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a3030e91531434f9bba4bb518ebf424'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:32.577845 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-84239728-ee5e-4206-9b05-213378058be6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a3030e91531434f9bba4bb518ebf424'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:32.579873 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-84239728-ee5e-4206-9b05-213378058be6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a3030e91531434f9bba4bb518ebf424'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:32.623142 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-84239728-ee5e-4206-9b05-213378058be6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a3030e91531434f9bba4bb518ebf424'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:32.652466 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 696/1390] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:05:32 2022] DELETE /identity/v3/users/9a3030e91531434f9bba4bb518ebf424 => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:05:32.656943 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e0010-3b28-4dd4-81f6-884e00525f14 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:32.657406 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e0010-3b28-4dd4-81f6-884e00525f14 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:32.657795 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485e0010-3b28-4dd4-81f6-884e00525f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:32.728002 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-485e0010-3b28-4dd4-81f6-884e00525f14 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:32.745135 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-485e0010-3b28-4dd4-81f6-884e00525f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:32.828869 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe4107-bca9-45bf-8dfe-3b34b327ea6c None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:32.829153 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe4107-bca9-45bf-8dfe-3b34b327ea6c None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:32.829430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abe4107-bca9-45bf-8dfe-3b34b327ea6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:32.872316 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:32.915731 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-7abe4107-bca9-45bf-8dfe-3b34b327ea6c None None] MFA Rules not processed for user `ba8ff3413bc549d29486d1c724a1f383`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:32.921457 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7abe4107-bca9-45bf-8dfe-3b34b327ea6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:32.999590 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 695/1391] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:33.024265 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e0010-3b28-4dd4-81f6-884e00525f14 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:33.025848 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-485e0010-3b28-4dd4-81f6-884e00525f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:33.112879 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 697/1392] 10.222.0.15 () {58 vars in 1015 bytes} [Sun Oct 23 20:05:32 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:05:33.127043 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe4107-bca9-45bf-8dfe-3b34b327ea6c None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:33.128956 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7abe4107-bca9-45bf-8dfe-3b34b327ea6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:33.150998 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abe4107-bca9-45bf-8dfe-3b34b327ea6c None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:33.175563 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485e0010-3b28-4dd4-81f6-884e00525f14 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:33.179524 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e429e84-4ae8-4df6-a172-c4558c7a9b04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:33.185726 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e429e84-4ae8-4df6-a172-c4558c7a9b04 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:33.186578 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e429e84-4ae8-4df6-a172-c4558c7a9b04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:33.186863 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e429e84-4ae8-4df6-a172-c4558c7a9b04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:33.190507 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd977062-acdb-4f2e-995c-79a9aea66c3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:33.191155 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd977062-acdb-4f2e-995c-79a9aea66c3a service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:33.191434 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd977062-acdb-4f2e-995c-79a9aea66c3a service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:33.191707 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd977062-acdb-4f2e-995c-79a9aea66c3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:33.219548 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e429e84-4ae8-4df6-a172-c4558c7a9b04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:33.222966 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e429e84-4ae8-4df6-a172-c4558c7a9b04 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:33.234070 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd977062-acdb-4f2e-995c-79a9aea66c3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:33.235774 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd977062-acdb-4f2e-995c-79a9aea66c3a service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:33.242651 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:33.258080 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 698/1393] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:33 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 20:05:33.273478 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 696/1394] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:05:35.495509 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e429e84-4ae8-4df6-a172-c4558c7a9b04 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:35.497135 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7e429e84-4ae8-4df6-a172-c4558c7a9b04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:35.524589 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e429e84-4ae8-4df6-a172-c4558c7a9b04 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:35.539221 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a831871-b7cf-48a8-9aa6-4c1072f00d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:35.539941 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a831871-b7cf-48a8-9aa6-4c1072f00d8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:35.541745 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a831871-b7cf-48a8-9aa6-4c1072f00d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:35.542044 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a831871-b7cf-48a8-9aa6-4c1072f00d8b admin admin] PATH_INFO: `/v3/users/8c82afa51f3a482ba72051e95fac2713` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:35.546777 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a831871-b7cf-48a8-9aa6-4c1072f00d8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c82afa51f3a482ba72051e95fac2713)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:35.548740 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a831871-b7cf-48a8-9aa6-4c1072f00d8b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:35.626933 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-3a831871-b7cf-48a8-9aa6-4c1072f00d8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c82afa51f3a482ba72051e95fac2713'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:35.642489 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-3a831871-b7cf-48a8-9aa6-4c1072f00d8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c82afa51f3a482ba72051e95fac2713'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:35.651841 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-3a831871-b7cf-48a8-9aa6-4c1072f00d8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c82afa51f3a482ba72051e95fac2713'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:35.652699 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-3a831871-b7cf-48a8-9aa6-4c1072f00d8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c82afa51f3a482ba72051e95fac2713'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:35.664510 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-3a831871-b7cf-48a8-9aa6-4c1072f00d8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c82afa51f3a482ba72051e95fac2713'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:35.681105 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 699/1395] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:05:35 2022] DELETE /identity/v3/users/8c82afa51f3a482ba72051e95fac2713 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:35.685876 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310bd83-6c1a-4169-b668-d44bc6ebc34e None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:35.686439 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310bd83-6c1a-4169-b668-d44bc6ebc34e None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:35.686609 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0310bd83-6c1a-4169-b668-d44bc6ebc34e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:35.718941 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-0310bd83-6c1a-4169-b668-d44bc6ebc34e None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:35.720600 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-0310bd83-6c1a-4169-b668-d44bc6ebc34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:35.801399 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 697/1396] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:35.822909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a831871-b7cf-48a8-9aa6-4c1072f00d8b admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:35.824611 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3a831871-b7cf-48a8-9aa6-4c1072f00d8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:35.949429 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a831871-b7cf-48a8-9aa6-4c1072f00d8b admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:35.965584 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb3ffb1-5496-405c-b84b-076bd6735ca6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:35.966602 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3ffb1-5496-405c-b84b-076bd6735ca6 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:35.967047 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3ffb1-5496-405c-b84b-076bd6735ca6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:35.967607 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb3ffb1-5496-405c-b84b-076bd6735ca6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:36.002394 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb3ffb1-5496-405c-b84b-076bd6735ca6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:36.008304 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb3ffb1-5496-405c-b84b-076bd6735ca6 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:36.010969 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310bd83-6c1a-4169-b668-d44bc6ebc34e None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:36.012316 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-0310bd83-6c1a-4169-b668-d44bc6ebc34e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:36.028744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0310bd83-6c1a-4169-b668-d44bc6ebc34e None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:36.037754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 700/1397] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:05:36.040916 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35a26e7-806a-4bd8-ae14-a5388c4b0554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:36.041926 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a26e7-806a-4bd8-ae14-a5388c4b0554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:36.042422 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a26e7-806a-4bd8-ae14-a5388c4b0554 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:36.042841 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35a26e7-806a-4bd8-ae14-a5388c4b0554 admin admin] PATH_INFO: `/v3/projects/ca2ed1e33c3b40a6b5c7aad91cffdbc7` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:36.044186 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35a26e7-806a-4bd8-ae14-a5388c4b0554 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca2ed1e33c3b40a6b5c7aad91cffdbc7)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:36.046291 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35a26e7-806a-4bd8-ae14-a5388c4b0554 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:36.125423 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c35a26e7-806a-4bd8-ae14-a5388c4b0554 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca2ed1e33c3b40a6b5c7aad91cffdbc7', 'request_id': 'req-c35a26e7-806a-4bd8-ae14-a5388c4b0554'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:36.135308 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c35a26e7-806a-4bd8-ae14-a5388c4b0554 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca2ed1e33c3b40a6b5c7aad91cffdbc7', 'request_id': 'req-c35a26e7-806a-4bd8-ae14-a5388c4b0554'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:36.144210 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c35a26e7-806a-4bd8-ae14-a5388c4b0554 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca2ed1e33c3b40a6b5c7aad91cffdbc7', 'request_id': 'req-c35a26e7-806a-4bd8-ae14-a5388c4b0554'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:36.161513 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c35a26e7-806a-4bd8-ae14-a5388c4b0554 admin admin] The token cache is being invalidate because project ca2ed1e33c3b40a6b5c7aad91cffdbc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:05:36.162071 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c35a26e7-806a-4bd8-ae14-a5388c4b0554 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:36.162788 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c35a26e7-806a-4bd8-ae14-a5388c4b0554 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:36.164730 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 698/1398] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:05:36 2022] DELETE /identity/v3/projects/ca2ed1e33c3b40a6b5c7aad91cffdbc7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:36.168440 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb3ffb1-5496-405c-b84b-076bd6735ca6 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:36.181145 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb3ffb1-5496-405c-b84b-076bd6735ca6 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:36.251388 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7719329-84a4-4dd9-846d-b1ed17116c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:36.252329 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7719329-84a4-4dd9-846d-b1ed17116c6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:36.252811 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7719329-84a4-4dd9-846d-b1ed17116c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:36.253804 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7719329-84a4-4dd9-846d-b1ed17116c6c admin admin] PATH_INFO: `/v3/users/a9ee4775d841485f8f38313bf5a90c0e` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:36.255639 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7719329-84a4-4dd9-846d-b1ed17116c6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9ee4775d841485f8f38313bf5a90c0e)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:36.258759 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7719329-84a4-4dd9-846d-b1ed17116c6c admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:36.281429 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:36.314103 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55efb8-eda7-423c-83bb-99a7d9117fd0 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:36.314427 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55efb8-eda7-423c-83bb-99a7d9117fd0 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:36.314736 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da55efb8-eda7-423c-83bb-99a7d9117fd0 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:36.316542 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 699/1399] 10.222.0.13 () {54 vars in 921 bytes} [Sun Oct 23 20:05:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:05:36.341335 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0b515-74d1-453c-b132-a66bfd1e91a5 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:36.342119 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0b515-74d1-453c-b132-a66bfd1e91a5 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:36.343274 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0b515-74d1-453c-b132-a66bfd1e91a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:36.356212 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-b7719329-84a4-4dd9-846d-b1ed17116c6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9ee4775d841485f8f38313bf5a90c0e'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:36.371103 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:36.374414 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-b7719329-84a4-4dd9-846d-b1ed17116c6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9ee4775d841485f8f38313bf5a90c0e'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:36.384585 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-b7719329-84a4-4dd9-846d-b1ed17116c6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9ee4775d841485f8f38313bf5a90c0e'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:36.385861 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-e9c0b515-74d1-453c-b132-a66bfd1e91a5 None None] MFA Rules not processed for user `0b3d512304254ffcaa8dad90abf1f9c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:36.386181 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-b7719329-84a4-4dd9-846d-b1ed17116c6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9ee4775d841485f8f38313bf5a90c0e'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:36.389056 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-e9c0b515-74d1-453c-b132-a66bfd1e91a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:36.397338 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-b7719329-84a4-4dd9-846d-b1ed17116c6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9ee4775d841485f8f38313bf5a90c0e'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:36.404616 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 701/1400] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:05:36 2022] DELETE /identity/v3/users/a9ee4775d841485f8f38313bf5a90c0e => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:36.484339 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 700/1401] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Oct 23 20:05:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:05:36.492275 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7719329-84a4-4dd9-846d-b1ed17116c6c admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:36.492953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-b7719329-84a4-4dd9-846d-b1ed17116c6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:36.544568 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0b515-74d1-453c-b132-a66bfd1e91a5 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:36.546175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-e9c0b515-74d1-453c-b132-a66bfd1e91a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:36.577898 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0b515-74d1-453c-b132-a66bfd1e91a5 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:36.658764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7719329-84a4-4dd9-846d-b1ed17116c6c admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:36.665541 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14647ceb-94f1-41d6-923e-fe3cf1cbe648 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:36.666173 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14647ceb-94f1-41d6-923e-fe3cf1cbe648 service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:36.666491 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14647ceb-94f1-41d6-923e-fe3cf1cbe648 service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:36.666771 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14647ceb-94f1-41d6-923e-fe3cf1cbe648 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:36.677688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14647ceb-94f1-41d6-923e-fe3cf1cbe648 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:36.680306 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14647ceb-94f1-41d6-923e-fe3cf1cbe648 service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:36.693352 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca7232-5ae1-4054-8d2f-b491023a0afc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:36.694487 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca7232-5ae1-4054-8d2f-b491023a0afc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:36.694723 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 702/1402] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:05:36.695198 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca7232-5ae1-4054-8d2f-b491023a0afc admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:36.695545 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bca7232-5ae1-4054-8d2f-b491023a0afc admin admin] PATH_INFO: `/v3/projects/9ba293d8c83844d28dd78b7d14c66771` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:36.696832 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bca7232-5ae1-4054-8d2f-b491023a0afc admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ba293d8c83844d28dd78b7d14c66771)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:36.699662 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bca7232-5ae1-4054-8d2f-b491023a0afc admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:36.794039 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-5bca7232-5ae1-4054-8d2f-b491023a0afc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ba293d8c83844d28dd78b7d14c66771', 'request_id': 'req-5bca7232-5ae1-4054-8d2f-b491023a0afc'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:36.807739 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-5bca7232-5ae1-4054-8d2f-b491023a0afc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ba293d8c83844d28dd78b7d14c66771', 'request_id': 'req-5bca7232-5ae1-4054-8d2f-b491023a0afc'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:36.818552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-5bca7232-5ae1-4054-8d2f-b491023a0afc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ba293d8c83844d28dd78b7d14c66771', 'request_id': 'req-5bca7232-5ae1-4054-8d2f-b491023a0afc'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:36.836619 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-5bca7232-5ae1-4054-8d2f-b491023a0afc admin admin] The token cache is being invalidate because project 9ba293d8c83844d28dd78b7d14c66771 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:05:36.836945 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-5bca7232-5ae1-4054-8d2f-b491023a0afc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:36.837622 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-5bca7232-5ae1-4054-8d2f-b491023a0afc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:36.839520 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 701/1403] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:05:36 2022] DELETE /identity/v3/projects/9ba293d8c83844d28dd78b7d14c66771 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:36.844265 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57c4f2-a2cd-44d9-a2c4-6fbd21b7ef48 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:36.844571 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57c4f2-a2cd-44d9-a2c4-6fbd21b7ef48 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:36.844830 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57c4f2-a2cd-44d9-a2c4-6fbd21b7ef48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:36.865614 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-1f57c4f2-a2cd-44d9-a2c4-6fbd21b7ef48 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:36.866942 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-1f57c4f2-a2cd-44d9-a2c4-6fbd21b7ef48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:36.874019 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca7232-5ae1-4054-8d2f-b491023a0afc admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:36.875765 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-5bca7232-5ae1-4054-8d2f-b491023a0afc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:36.910292 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:36.945218 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 703/1404] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:36.952376 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1305e18-43ce-4af4-8b07-78ca7985ed8f None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:36.952763 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1305e18-43ce-4af4-8b07-78ca7985ed8f None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:36.953043 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1305e18-43ce-4af4-8b07-78ca7985ed8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:36.959203 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bca7232-5ae1-4054-8d2f-b491023a0afc admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:36.965992 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4890d2e-fd12-415a-a8e4-3f6924c8ed1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:36.966670 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4890d2e-fd12-415a-a8e4-3f6924c8ed1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:36.966952 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4890d2e-fd12-415a-a8e4-3f6924c8ed1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:36.967227 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4890d2e-fd12-415a-a8e4-3f6924c8ed1a admin admin] PATH_INFO: `/v3/projects/daa4d2797ae545b78d9e7daefe273410` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:36.968813 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4890d2e-fd12-415a-a8e4-3f6924c8ed1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=daa4d2797ae545b78d9e7daefe273410)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:36.970938 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4890d2e-fd12-415a-a8e4-3f6924c8ed1a admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:36.974687 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-a1305e18-43ce-4af4-8b07-78ca7985ed8f None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:36.976518 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a1305e18-43ce-4af4-8b07-78ca7985ed8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:36.984447 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 704/1405] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:05:36.988679 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1305e18-43ce-4af4-8b07-78ca7985ed8f None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:36.999539 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1305e18-43ce-4af4-8b07-78ca7985ed8f None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:37.003875 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70251446-38d6-4403-9d9e-a7ea11b4cf38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:37.004426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70251446-38d6-4403-9d9e-a7ea11b4cf38 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:37.004720 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70251446-38d6-4403-9d9e-a7ea11b4cf38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:37.004991 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70251446-38d6-4403-9d9e-a7ea11b4cf38 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:37.005625 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70251446-38d6-4403-9d9e-a7ea11b4cf38 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:05:37.006222 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70251446-38d6-4403-9d9e-a7ea11b4cf38 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:37.009075 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70251446-38d6-4403-9d9e-a7ea11b4cf38 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:37.019315 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:37.037134 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 705/1406] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:05:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 20:05:37.037713 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-a4890d2e-fd12-415a-a8e4-3f6924c8ed1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daa4d2797ae545b78d9e7daefe273410', 'request_id': 'req-a4890d2e-fd12-415a-a8e4-3f6924c8ed1a'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:37.043617 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70251446-38d6-4403-9d9e-a7ea11b4cf38 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:37.051887 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-a4890d2e-fd12-415a-a8e4-3f6924c8ed1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daa4d2797ae545b78d9e7daefe273410', 'request_id': 'req-a4890d2e-fd12-415a-a8e4-3f6924c8ed1a'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:37.056852 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70251446-38d6-4403-9d9e-a7ea11b4cf38 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:37.060801 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-a4890d2e-fd12-415a-a8e4-3f6924c8ed1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daa4d2797ae545b78d9e7daefe273410', 'request_id': 'req-a4890d2e-fd12-415a-a8e4-3f6924c8ed1a'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:37.079762 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-a4890d2e-fd12-415a-a8e4-3f6924c8ed1a admin admin] The token cache is being invalidate because project daa4d2797ae545b78d9e7daefe273410 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:05:37.080149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-a4890d2e-fd12-415a-a8e4-3f6924c8ed1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:37.080897 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-a4890d2e-fd12-415a-a8e4-3f6924c8ed1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:37.082919 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 702/1407] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:05:36 2022] DELETE /identity/v3/projects/daa4d2797ae545b78d9e7daefe273410 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:05:37.087472 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b523f-6cc0-4e55-80e4-f462ab2f50f2 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:37.087885 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b523f-6cc0-4e55-80e4-f462ab2f50f2 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:37.088258 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0b523f-6cc0-4e55-80e4-f462ab2f50f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:37.116384 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-ea0b523f-6cc0-4e55-80e4-f462ab2f50f2 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:37.117775 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-ea0b523f-6cc0-4e55-80e4-f462ab2f50f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:37.147234 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0fa8e0-8fbe-4bbc-a80c-64b0896e38ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:37.148064 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fa8e0-8fbe-4bbc-a80c-64b0896e38ea admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:37.148352 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fa8e0-8fbe-4bbc-a80c-64b0896e38ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:37.148630 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fa8e0-8fbe-4bbc-a80c-64b0896e38ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:37.149325 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0fa8e0-8fbe-4bbc-a80c-64b0896e38ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:37.151421 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0fa8e0-8fbe-4bbc-a80c-64b0896e38ea admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:37.183825 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 706/1408] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:05:37 2022] POST /identity/v3/projects => generated 401 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:37.187880 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0fa8e0-8fbe-4bbc-a80c-64b0896e38ea admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:37.189509 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-cf0fa8e0-8fbe-4bbc-a80c-64b0896e38ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:37.200821 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 703/1409] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:37.208429 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c645f7-5b18-4519-8817-de3838c2e434 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:37.208717 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c645f7-5b18-4519-8817-de3838c2e434 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:37.208994 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c645f7-5b18-4519-8817-de3838c2e434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:37.235347 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-a1c645f7-5b18-4519-8817-de3838c2e434 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:37.236618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-a1c645f7-5b18-4519-8817-de3838c2e434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:37.265419 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0fa8e0-8fbe-4bbc-a80c-64b0896e38ea admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:37.270202 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8655b461-bd18-415e-9255-439fc8c1a9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:37.270847 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655b461-bd18-415e-9255-439fc8c1a9fb admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:37.271128 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655b461-bd18-415e-9255-439fc8c1a9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:37.271400 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8655b461-bd18-415e-9255-439fc8c1a9fb admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:37.272037 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8655b461-bd18-415e-9255-439fc8c1a9fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:37.274163 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8655b461-bd18-415e-9255-439fc8c1a9fb admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:37.302592 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 707/1410] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:05:37 2022] POST /identity/v3/users => generated 301 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:05:37.303620 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8655b461-bd18-415e-9255-439fc8c1a9fb admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:37.320784 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8655b461-bd18-415e-9255-439fc8c1a9fb admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:37.323702 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:37.334128 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3716d1f-401b-4878-a8b8-0ea8a663bc40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:37.334610 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 704/1411] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:37.334910 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3716d1f-401b-4878-a8b8-0ea8a663bc40 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:37.335252 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3716d1f-401b-4878-a8b8-0ea8a663bc40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:37.335636 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3716d1f-401b-4878-a8b8-0ea8a663bc40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:37.336402 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3716d1f-401b-4878-a8b8-0ea8a663bc40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:37.338592 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c645f7-5b18-4519-8817-de3838c2e434 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:37.338729 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3716d1f-401b-4878-a8b8-0ea8a663bc40 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:37.350954 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c645f7-5b18-4519-8817-de3838c2e434 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:37.354546 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 708/1412] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 20:05:37.356451 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3716d1f-401b-4878-a8b8-0ea8a663bc40 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:37.369750 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15e65f9-2d12-49be-8372-2b2b1268d94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:37.370405 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e65f9-2d12-49be-8372-2b2b1268d94f admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:37.370793 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e65f9-2d12-49be-8372-2b2b1268d94f admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:37.371074 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15e65f9-2d12-49be-8372-2b2b1268d94f admin admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:37.371738 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15e65f9-2d12-49be-8372-2b2b1268d94f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:05:37.372048 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15e65f9-2d12-49be-8372-2b2b1268d94f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:37.373842 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15e65f9-2d12-49be-8372-2b2b1268d94f admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:37.383782 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3716d1f-401b-4878-a8b8-0ea8a663bc40 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:37.392995 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 705/1413] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:05:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:05:37.394068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15e65f9-2d12-49be-8372-2b2b1268d94f admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:37.396353 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d3522-6264-4843-aefd-f0b148a3aabf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:37.399628 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d3522-6264-4843-aefd-f0b148a3aabf admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:37.400474 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d3522-6264-4843-aefd-f0b148a3aabf admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:37.403854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0d3522-6264-4843-aefd-f0b148a3aabf admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:37.405403 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15e65f9-2d12-49be-8372-2b2b1268d94f admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:37.405794 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0d3522-6264-4843-aefd-f0b148a3aabf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:37.410004 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0d3522-6264-4843-aefd-f0b148a3aabf admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:37.410370 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1cd70c-b734-4c43-9a38-1c2661b51794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:37.411000 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1cd70c-b734-4c43-9a38-1c2661b51794 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:37.411290 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1cd70c-b734-4c43-9a38-1c2661b51794 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:37.411555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1cd70c-b734-4c43-9a38-1c2661b51794 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:37.412205 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1cd70c-b734-4c43-9a38-1c2661b51794 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:37.414903 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1cd70c-b734-4c43-9a38-1c2661b51794 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:37.445298 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 709/1414] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:37.450328 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d3522-6264-4843-aefd-f0b148a3aabf admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:37.458520 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 706/1415] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:05:37 2022] POST /identity/v3/projects => generated 389 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:37.462213 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1cd70c-b734-4c43-9a38-1c2661b51794 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:37.468312 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0d3522-6264-4843-aefd-f0b148a3aabf admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:37.550338 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:37.576633 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac7afc-f1c2-410d-9658-17f5435bd0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:37.577332 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac7afc-f1c2-410d-9658-17f5435bd0e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:37.577686 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac7afc-f1c2-410d-9658-17f5435bd0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:37.578029 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac7afc-f1c2-410d-9658-17f5435bd0e5 admin admin] PATH_INFO: `/v3/projects/42ceda910739400390f5074a7c8f0e9d/users/a5a9fe7502cf4f1d8a4c6406e2ff77f8/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:37.578126 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1cd70c-b734-4c43-9a38-1c2661b51794 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:37.584008 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ad6de-d8a8-43d1-82db-a99a25fba4c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:37.584604 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ad6de-d8a8-43d1-82db-a99a25fba4c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:37.584880 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ad6de-d8a8-43d1-82db-a99a25fba4c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:37.585962 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ad6de-d8a8-43d1-82db-a99a25fba4c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:37.586816 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ad6de-d8a8-43d1-82db-a99a25fba4c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:37.597534 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac7afc-f1c2-410d-9658-17f5435bd0e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42ceda910739400390f5074a7c8f0e9d, user_id=a5a9fe7502cf4f1d8a4c6406e2ff77f8, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:37.599860 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ad6de-d8a8-43d1-82db-a99a25fba4c1 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:37.600295 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac7afc-f1c2-410d-9658-17f5435bd0e5 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:37.634729 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 707/1416] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:05:37 2022] POST /identity/v3/users => generated 295 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:05:37.639673 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ad6de-d8a8-43d1-82db-a99a25fba4c1 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:37.650288 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 710/1417] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:37 2022] PUT /identity/v3/projects/42ceda910739400390f5074a7c8f0e9d/users/a5a9fe7502cf4f1d8a4c6406e2ff77f8/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:37.653592 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f0cc7-2520-4f46-8694-9e943a19caee None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:37.653737 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f0cc7-2520-4f46-8694-9e943a19caee None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:37.654001 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5f0cc7-2520-4f46-8694-9e943a19caee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:37.668423 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ad6de-d8a8-43d1-82db-a99a25fba4c1 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:37.699250 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-ca5f0cc7-2520-4f46-8694-9e943a19caee None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:37.701161 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ca5f0cc7-2520-4f46-8694-9e943a19caee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:37.835878 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccafa374-444e-456b-8118-016afab968b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:37.836444 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 711/1418] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:37.836719 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafa374-444e-456b-8118-016afab968b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:37.836837 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafa374-444e-456b-8118-016afab968b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:37.837117 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafa374-444e-456b-8118-016afab968b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:37.837791 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccafa374-444e-456b-8118-016afab968b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:37.839676 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccafa374-444e-456b-8118-016afab968b7 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:37.849231 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f0cc7-2520-4f46-8694-9e943a19caee None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:37.851781 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ca5f0cc7-2520-4f46-8694-9e943a19caee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:37.866272 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 708/1419] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 20:05:37.867476 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccafa374-444e-456b-8118-016afab968b7 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:37.886286 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccafa374-444e-456b-8118-016afab968b7 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:37.901063 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e8ac7-99f2-478a-a56f-4cb62b4bcb0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:37.901626 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e8ac7-99f2-478a-a56f-4cb62b4bcb0d admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:37.901920 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e8ac7-99f2-478a-a56f-4cb62b4bcb0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:37.902180 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e8ac7-99f2-478a-a56f-4cb62b4bcb0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:37.903134 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e8ac7-99f2-478a-a56f-4cb62b4bcb0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:37.904752 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7e8ac7-99f2-478a-a56f-4cb62b4bcb0d admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:37.926498 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 709/1420] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:37 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 20:05:37.931257 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e8ac7-99f2-478a-a56f-4cb62b4bcb0d admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:37.958400 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7e8ac7-99f2-478a-a56f-4cb62b4bcb0d admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:37.965246 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179d3faa-93c1-4c47-ae6d-1516c44d28a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:37.965921 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d3faa-93c1-4c47-ae6d-1516c44d28a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:37.966199 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d3faa-93c1-4c47-ae6d-1516c44d28a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:37.966505 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179d3faa-93c1-4c47-ae6d-1516c44d28a2 admin admin] PATH_INFO: `/v3/projects/063f2189a96c40b8b88fac854620ba90/users/588291953e654642bd3fbec90ea17366/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:37.973439 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5f0cc7-2520-4f46-8694-9e943a19caee None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:37.978870 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649558b-a86a-47fb-b967-7c54c5feefbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:37.979757 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649558b-a86a-47fb-b967-7c54c5feefbb service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:37.980040 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649558b-a86a-47fb-b967-7c54c5feefbb service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:37.980442 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1649558b-a86a-47fb-b967-7c54c5feefbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:37.983414 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d3faa-93c1-4c47-ae6d-1516c44d28a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=063f2189a96c40b8b88fac854620ba90, user_id=588291953e654642bd3fbec90ea17366, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:37.986269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179d3faa-93c1-4c47-ae6d-1516c44d28a2 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:37.992987 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1649558b-a86a-47fb-b967-7c54c5feefbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:37.994586 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1649558b-a86a-47fb-b967-7c54c5feefbb service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:38.001390 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 710/1421] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:37 2022] PUT /identity/v3/projects/063f2189a96c40b8b88fac854620ba90/users/588291953e654642bd3fbec90ea17366/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:38.005925 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 712/1422] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 20:05:38.011984 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855baf78-5234-4e75-9c73-1cfdf68dec82 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:38.012339 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855baf78-5234-4e75-9c73-1cfdf68dec82 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:38.012539 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855baf78-5234-4e75-9c73-1cfdf68dec82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:38.051390 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-855baf78-5234-4e75-9c73-1cfdf68dec82 None None] MFA Rules not processed for user `588291953e654642bd3fbec90ea17366`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:38.052743 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-855baf78-5234-4e75-9c73-1cfdf68dec82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:38.057949 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:38.175290 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 711/1423] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:38 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:38.176799 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649558b-a86a-47fb-b967-7c54c5feefbb service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:38.189304 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1649558b-a86a-47fb-b967-7c54c5feefbb service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:38.267007 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30f64bc-c5b4-4b60-8a72-7a1836ee1cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:38.267640 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f64bc-c5b4-4b60-8a72-7a1836ee1cbc admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:38.267929 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f64bc-c5b4-4b60-8a72-7a1836ee1cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:38.268207 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30f64bc-c5b4-4b60-8a72-7a1836ee1cbc admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:38.268865 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30f64bc-c5b4-4b60-8a72-7a1836ee1cbc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:38.270801 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30f64bc-c5b4-4b60-8a72-7a1836ee1cbc admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:38.338116 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 713/1424] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:05:38 2022] POST /identity/v3/projects => generated 389 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:38.339280 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855baf78-5234-4e75-9c73-1cfdf68dec82 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:38.363661 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855baf78-5234-4e75-9c73-1cfdf68dec82 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:38.444015 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60dc500-1e38-48ff-bf2e-57fd018277e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:38.445206 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60dc500-1e38-48ff-bf2e-57fd018277e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:38.445520 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60dc500-1e38-48ff-bf2e-57fd018277e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:38.445887 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60dc500-1e38-48ff-bf2e-57fd018277e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:38.446875 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60dc500-1e38-48ff-bf2e-57fd018277e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:38.448824 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60dc500-1e38-48ff-bf2e-57fd018277e4 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:38.486585 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 712/1425] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:05:38 2022] POST /identity/v3/users => generated 301 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:05:38.488122 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30f64bc-c5b4-4b60-8a72-7a1836ee1cbc admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:38.498478 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:38.504645 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30f64bc-c5b4-4b60-8a72-7a1836ee1cbc admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:38.512459 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c2e36-fea7-4a47-b0ce-cf183264a057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:38.515847 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c2e36-fea7-4a47-b0ce-cf183264a057 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:38.516054 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c2e36-fea7-4a47-b0ce-cf183264a057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:38.516327 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1c2e36-fea7-4a47-b0ce-cf183264a057 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:38.517002 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c2e36-fea7-4a47-b0ce-cf183264a057 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:38.519293 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1c2e36-fea7-4a47-b0ce-cf183264a057 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:38.533752 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 714/1426] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:38.537785 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60dc500-1e38-48ff-bf2e-57fd018277e4 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:38.560385 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60dc500-1e38-48ff-bf2e-57fd018277e4 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:38.567811 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d7f7bf-49d5-4d22-bdc3-6fa719a13006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:38.568627 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7f7bf-49d5-4d22-bdc3-6fa719a13006 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:38.569311 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7f7bf-49d5-4d22-bdc3-6fa719a13006 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:38.569722 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7f7bf-49d5-4d22-bdc3-6fa719a13006 admin admin] PATH_INFO: `/v3/projects/ff8e17bfb2214ba48e9b6dd740a1dd06/users/9b551c00bcf14c9e97d4dd1a0ad55349/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:38.601921 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d7f7bf-49d5-4d22-bdc3-6fa719a13006 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff8e17bfb2214ba48e9b6dd740a1dd06, user_id=9b551c00bcf14c9e97d4dd1a0ad55349, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:38.604664 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d7f7bf-49d5-4d22-bdc3-6fa719a13006 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:38.622002 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 713/1427] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:38 2022] PUT /identity/v3/projects/ff8e17bfb2214ba48e9b6dd740a1dd06/users/9b551c00bcf14c9e97d4dd1a0ad55349/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:05:38.626648 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c2e36-fea7-4a47-b0ce-cf183264a057 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:38.639658 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1c2e36-fea7-4a47-b0ce-cf183264a057 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:38.698769 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08561927-41c0-4957-9e67-5fb15d41f6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:38.699453 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08561927-41c0-4957-9e67-5fb15d41f6a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:38.700016 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08561927-41c0-4957-9e67-5fb15d41f6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:38.700016 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08561927-41c0-4957-9e67-5fb15d41f6a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:38.700635 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08561927-41c0-4957-9e67-5fb15d41f6a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:38.702205 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08561927-41c0-4957-9e67-5fb15d41f6a6 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:38.712473 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 715/1428] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:38 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:38.716966 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d7f7bf-49d5-4d22-bdc3-6fa719a13006 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:38.727658 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d7f7bf-49d5-4d22-bdc3-6fa719a13006 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:38.732807 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e41bd-a685-4e8c-9ff7-6a30aa53b2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:38.733643 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e41bd-a685-4e8c-9ff7-6a30aa53b2cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:38.733927 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e41bd-a685-4e8c-9ff7-6a30aa53b2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:38.734198 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e41bd-a685-4e8c-9ff7-6a30aa53b2cf admin admin] PATH_INFO: `/v3/projects/ff8e17bfb2214ba48e9b6dd740a1dd06/users/9b551c00bcf14c9e97d4dd1a0ad55349/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:38.736496 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375e41bd-a685-4e8c-9ff7-6a30aa53b2cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff8e17bfb2214ba48e9b6dd740a1dd06, user_id=9b551c00bcf14c9e97d4dd1a0ad55349, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:38.738901 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375e41bd-a685-4e8c-9ff7-6a30aa53b2cf admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:38.749767 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 714/1429] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:38 2022] PUT /identity/v3/projects/ff8e17bfb2214ba48e9b6dd740a1dd06/users/9b551c00bcf14c9e97d4dd1a0ad55349/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:38.763032 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e32904-0530-4bd8-8765-07bc97681252 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:38.763823 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e32904-0530-4bd8-8765-07bc97681252 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:38.764371 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e32904-0530-4bd8-8765-07bc97681252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:38.809734 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-10e32904-0530-4bd8-8765-07bc97681252 None None] MFA Rules not processed for user `9b551c00bcf14c9e97d4dd1a0ad55349`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:38.811124 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-10e32904-0530-4bd8-8765-07bc97681252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:38.880007 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386a7d9-09a0-4ddc-9f21-142b110383eb None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:38.880463 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386a7d9-09a0-4ddc-9f21-142b110383eb None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:38.880579 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9386a7d9-09a0-4ddc-9f21-142b110383eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:38.918837 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-9386a7d9-09a0-4ddc-9f21-142b110383eb None None] MFA Rules not processed for user `a5a9fe7502cf4f1d8a4c6406e2ff77f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:38.920006 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-9386a7d9-09a0-4ddc-9f21-142b110383eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:38.931718 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 716/1430] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:38 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:05:38.943301 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e32904-0530-4bd8-8765-07bc97681252 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:38.947174 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-10e32904-0530-4bd8-8765-07bc97681252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:38.979659 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:39.024118 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e32904-0530-4bd8-8765-07bc97681252 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:39.030537 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934e1569-34d1-4d5f-8453-f97ea00784bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:39.030771 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934e1569-34d1-4d5f-8453-f97ea00784bd service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:39.031097 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934e1569-34d1-4d5f-8453-f97ea00784bd service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:39.031383 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934e1569-34d1-4d5f-8453-f97ea00784bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:39.036507 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 715/1431] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:38 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:39.037959 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386a7d9-09a0-4ddc-9f21-142b110383eb None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:39.054558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934e1569-34d1-4d5f-8453-f97ea00784bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:39.056027 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934e1569-34d1-4d5f-8453-f97ea00784bd service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:39.078232 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 717/1432] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:38 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:05:39.170846 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9386a7d9-09a0-4ddc-9f21-142b110383eb None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:39.187650 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db91f7a-e1c2-45ad-a52a-a348eea7f751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:39.188442 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db91f7a-e1c2-45ad-a52a-a348eea7f751 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:39.188730 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db91f7a-e1c2-45ad-a52a-a348eea7f751 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:39.189005 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db91f7a-e1c2-45ad-a52a-a348eea7f751 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:39.189657 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db91f7a-e1c2-45ad-a52a-a348eea7f751 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:39.196478 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db91f7a-e1c2-45ad-a52a-a348eea7f751 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:39.263684 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 716/1433] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:05:39 2022] POST /identity/v3/projects => generated 403 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:39.264878 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934e1569-34d1-4d5f-8453-f97ea00784bd service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:39.282145 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934e1569-34d1-4d5f-8453-f97ea00784bd service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:39.292464 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db91f7a-e1c2-45ad-a52a-a348eea7f751 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:39.310895 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:39.406774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee44dc3b-9611-4944-a01b-b63b2648a9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:39.408760 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44dc3b-9611-4944-a01b-b63b2648a9ad admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:39.409353 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44dc3b-9611-4944-a01b-b63b2648a9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:39.409983 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee44dc3b-9611-4944-a01b-b63b2648a9ad admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:39.410815 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee44dc3b-9611-4944-a01b-b63b2648a9ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:39.422187 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee44dc3b-9611-4944-a01b-b63b2648a9ad admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:39.429033 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db91f7a-e1c2-45ad-a52a-a348eea7f751 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:39.442052 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582c4143-a12e-4905-8907-3e97eba48651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:39.443009 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c4143-a12e-4905-8907-3e97eba48651 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:39.443444 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c4143-a12e-4905-8907-3e97eba48651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:39.444128 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582c4143-a12e-4905-8907-3e97eba48651 admin admin] PATH_INFO: `/v3/users/588291953e654642bd3fbec90ea17366` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:39.449762 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582c4143-a12e-4905-8907-3e97eba48651 admin admin] RBAC: Authorizing `identity:delete_user(user_id=588291953e654642bd3fbec90ea17366)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:39.451847 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582c4143-a12e-4905-8907-3e97eba48651 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:39.483321 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 718/1434] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:05:39 2022] POST /identity/v3/users => generated 308 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:39.484631 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee44dc3b-9611-4944-a01b-b63b2648a9ad admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:39.498452 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee44dc3b-9611-4944-a01b-b63b2648a9ad admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:39.506169 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d56824-a526-4a2e-8316-35a66837887b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:39.507557 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d56824-a526-4a2e-8316-35a66837887b admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:39.508865 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d56824-a526-4a2e-8316-35a66837887b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:39.509155 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d56824-a526-4a2e-8316-35a66837887b admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:39.509870 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d56824-a526-4a2e-8316-35a66837887b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:39.511675 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d56824-a526-4a2e-8316-35a66837887b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:39.535907 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 719/1435] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:39.542064 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d56824-a526-4a2e-8316-35a66837887b admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:39.554660 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d56824-a526-4a2e-8316-35a66837887b admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:39.559749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006bfa6a-60b2-4ded-83f4-a7bab81ff6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:39.560558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bfa6a-60b2-4ded-83f4-a7bab81ff6c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:39.561255 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bfa6a-60b2-4ded-83f4-a7bab81ff6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:39.561615 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006bfa6a-60b2-4ded-83f4-a7bab81ff6c6 admin admin] PATH_INFO: `/v3/projects/9086a333071641999c47ccbeb4df233f/users/ad67526a2e6d4d3bba96fc2f6405b756/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:39.570758 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-582c4143-a12e-4905-8907-3e97eba48651 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '588291953e654642bd3fbec90ea17366'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:39.583583 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006bfa6a-60b2-4ded-83f4-a7bab81ff6c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9086a333071641999c47ccbeb4df233f, user_id=ad67526a2e6d4d3bba96fc2f6405b756, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:39.587104 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006bfa6a-60b2-4ded-83f4-a7bab81ff6c6 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:39.605502 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 720/1436] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:39 2022] PUT /identity/v3/projects/9086a333071641999c47ccbeb4df233f/users/ad67526a2e6d4d3bba96fc2f6405b756/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:05:39.610538 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006bfa6a-60b2-4ded-83f4-a7bab81ff6c6 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:39.612390 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-582c4143-a12e-4905-8907-3e97eba48651 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '588291953e654642bd3fbec90ea17366'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:39.631125 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-582c4143-a12e-4905-8907-3e97eba48651 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '588291953e654642bd3fbec90ea17366'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:39.637116 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:39.645077 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-582c4143-a12e-4905-8907-3e97eba48651 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '588291953e654642bd3fbec90ea17366'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:39.645989 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-582c4143-a12e-4905-8907-3e97eba48651 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '588291953e654642bd3fbec90ea17366'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:39.659425 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 717/1437] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:05:39 2022] DELETE /identity/v3/users/588291953e654642bd3fbec90ea17366 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:39.666754 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24fbe8-efb7-4eb3-b3c2-01845496c76e None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:39.667185 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24fbe8-efb7-4eb3-b3c2-01845496c76e None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:39.667520 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b24fbe8-efb7-4eb3-b3c2-01845496c76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:39.699808 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-4b24fbe8-efb7-4eb3-b3c2-01845496c76e None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:39.701055 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-4b24fbe8-efb7-4eb3-b3c2-01845496c76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:39.724139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006bfa6a-60b2-4ded-83f4-a7bab81ff6c6 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:39.793358 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29eb2f3-e2b8-4b0f-86a0-4e4fb6d44054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:39.794314 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29eb2f3-e2b8-4b0f-86a0-4e4fb6d44054 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:39.794597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29eb2f3-e2b8-4b0f-86a0-4e4fb6d44054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:39.794944 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29eb2f3-e2b8-4b0f-86a0-4e4fb6d44054 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:39.795598 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29eb2f3-e2b8-4b0f-86a0-4e4fb6d44054 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:39.798017 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29eb2f3-e2b8-4b0f-86a0-4e4fb6d44054 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:39.815715 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 721/1438] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:39 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:39.825424 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29eb2f3-e2b8-4b0f-86a0-4e4fb6d44054 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:39.827391 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d29eb2f3-e2b8-4b0f-86a0-4e4fb6d44054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:39.827993 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 718/1439] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:39.851794 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24fbe8-efb7-4eb3-b3c2-01845496c76e None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:39.853706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29eb2f3-e2b8-4b0f-86a0-4e4fb6d44054 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:39.854931 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-4b24fbe8-efb7-4eb3-b3c2-01845496c76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:39.859423 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faaa731-fba1-48b5-b876-9b2e05b59762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:39.860434 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faaa731-fba1-48b5-b876-9b2e05b59762 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:39.861030 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faaa731-fba1-48b5-b876-9b2e05b59762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:39.861456 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faaa731-fba1-48b5-b876-9b2e05b59762 admin admin] PATH_INFO: `/v3/projects/9086a333071641999c47ccbeb4df233f/users/ad67526a2e6d4d3bba96fc2f6405b756/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:39.863592 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faaa731-fba1-48b5-b876-9b2e05b59762 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9086a333071641999c47ccbeb4df233f, user_id=ad67526a2e6d4d3bba96fc2f6405b756, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:39.867692 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faaa731-fba1-48b5-b876-9b2e05b59762 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:39.879920 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 722/1440] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:39 2022] PUT /identity/v3/projects/9086a333071641999c47ccbeb4df233f/users/ad67526a2e6d4d3bba96fc2f6405b756/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:39.952719 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b24fbe8-efb7-4eb3-b3c2-01845496c76e None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:40.029099 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8b9a1-c916-4e1d-a8cb-96bd746809cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:40.029740 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8b9a1-c916-4e1d-a8cb-96bd746809cb service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:40.030020 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8b9a1-c916-4e1d-a8cb-96bd746809cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:40.030380 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8b9a1-c916-4e1d-a8cb-96bd746809cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:40.131255 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8b9a1-c916-4e1d-a8cb-96bd746809cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:40.133780 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8b9a1-c916-4e1d-a8cb-96bd746809cb service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:40.137861 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:40.147499 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 719/1441] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:05:40.542158 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faaa731-fba1-48b5-b876-9b2e05b59762 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:40.547779 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-0faaa731-fba1-48b5-b876-9b2e05b59762 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:40.577585 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:40.609765 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faaa731-fba1-48b5-b876-9b2e05b59762 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:40.619007 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89c9c4a-db6e-4b99-a6ae-ae21a624fb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:40.626132 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c9c4a-db6e-4b99-a6ae-ae21a624fb4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:40.626637 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c9c4a-db6e-4b99-a6ae-ae21a624fb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:40.627242 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89c9c4a-db6e-4b99-a6ae-ae21a624fb4a admin admin] PATH_INFO: `/v3/projects/063f2189a96c40b8b88fac854620ba90` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:40.628997 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89c9c4a-db6e-4b99-a6ae-ae21a624fb4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=063f2189a96c40b8b88fac854620ba90)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:40.632029 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89c9c4a-db6e-4b99-a6ae-ae21a624fb4a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:40.754458 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e89c9c4a-db6e-4b99-a6ae-ae21a624fb4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '063f2189a96c40b8b88fac854620ba90', 'request_id': 'req-e89c9c4a-db6e-4b99-a6ae-ae21a624fb4a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:40.762912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d39e7-4a9a-4f6a-998b-e7d910d2a3a5 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:40.763251 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d39e7-4a9a-4f6a-998b-e7d910d2a3a5 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:40.763536 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d39e7-4a9a-4f6a-998b-e7d910d2a3a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:40.777838 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e89c9c4a-db6e-4b99-a6ae-ae21a624fb4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '063f2189a96c40b8b88fac854620ba90', 'request_id': 'req-e89c9c4a-db6e-4b99-a6ae-ae21a624fb4a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:40.793717 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e89c9c4a-db6e-4b99-a6ae-ae21a624fb4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '063f2189a96c40b8b88fac854620ba90', 'request_id': 'req-e89c9c4a-db6e-4b99-a6ae-ae21a624fb4a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:40.813297 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e89c9c4a-db6e-4b99-a6ae-ae21a624fb4a admin admin] The token cache is being invalidate because project 063f2189a96c40b8b88fac854620ba90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:05:40.813925 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e89c9c4a-db6e-4b99-a6ae-ae21a624fb4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:40.814909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e89c9c4a-db6e-4b99-a6ae-ae21a624fb4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:40.817435 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 723/1442] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:05:40 2022] DELETE /identity/v3/projects/063f2189a96c40b8b88fac854620ba90 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:40.821898 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89c9c4a-db6e-4b99-a6ae-ae21a624fb4a admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:40.825231 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e89c9c4a-db6e-4b99-a6ae-ae21a624fb4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:40.840574 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-132d39e7-4a9a-4f6a-998b-e7d910d2a3a5 None None] MFA Rules not processed for user `ad67526a2e6d4d3bba96fc2f6405b756`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:40.844640 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-132d39e7-4a9a-4f6a-998b-e7d910d2a3a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:41.002973 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89c9c4a-db6e-4b99-a6ae-ae21a624fb4a admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:41.031257 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825a9483-cfaf-4501-95c3-2a58784c0d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:41.032200 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a9483-cfaf-4501-95c3-2a58784c0d84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:41.033100 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a9483-cfaf-4501-95c3-2a58784c0d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:41.033680 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825a9483-cfaf-4501-95c3-2a58784c0d84 admin admin] PATH_INFO: `/v3/users/9b551c00bcf14c9e97d4dd1a0ad55349` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:41.036158 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a9483-cfaf-4501-95c3-2a58784c0d84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b551c00bcf14c9e97d4dd1a0ad55349)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:41.038804 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825a9483-cfaf-4501-95c3-2a58784c0d84 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:41.087934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 720/1443] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:40 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:41.176871 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-825a9483-cfaf-4501-95c3-2a58784c0d84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b551c00bcf14c9e97d4dd1a0ad55349'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:41.206789 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-825a9483-cfaf-4501-95c3-2a58784c0d84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b551c00bcf14c9e97d4dd1a0ad55349'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:41.221040 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-825a9483-cfaf-4501-95c3-2a58784c0d84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b551c00bcf14c9e97d4dd1a0ad55349'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:41.222335 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-825a9483-cfaf-4501-95c3-2a58784c0d84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b551c00bcf14c9e97d4dd1a0ad55349'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:41.236692 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-825a9483-cfaf-4501-95c3-2a58784c0d84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b551c00bcf14c9e97d4dd1a0ad55349'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:41.248504 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 724/1444] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:05:40 2022] DELETE /identity/v3/users/9b551c00bcf14c9e97d4dd1a0ad55349 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:42.247042 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d39e7-4a9a-4f6a-998b-e7d910d2a3a5 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:42.268841 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132d39e7-4a9a-4f6a-998b-e7d910d2a3a5 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:42.498552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175be21e-bf32-4216-9be2-f584b57516e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:42.499490 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175be21e-bf32-4216-9be2-f584b57516e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:42.499780 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175be21e-bf32-4216-9be2-f584b57516e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:42.500055 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175be21e-bf32-4216-9be2-f584b57516e9 admin admin] PATH_INFO: `/v3/projects/ff8e17bfb2214ba48e9b6dd740a1dd06` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:42.501332 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175be21e-bf32-4216-9be2-f584b57516e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff8e17bfb2214ba48e9b6dd740a1dd06)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:42.504679 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175be21e-bf32-4216-9be2-f584b57516e9 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:42.599642 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-175be21e-bf32-4216-9be2-f584b57516e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff8e17bfb2214ba48e9b6dd740a1dd06', 'request_id': 'req-175be21e-bf32-4216-9be2-f584b57516e9'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:42.612701 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-175be21e-bf32-4216-9be2-f584b57516e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff8e17bfb2214ba48e9b6dd740a1dd06', 'request_id': 'req-175be21e-bf32-4216-9be2-f584b57516e9'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:42.637169 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-175be21e-bf32-4216-9be2-f584b57516e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff8e17bfb2214ba48e9b6dd740a1dd06', 'request_id': 'req-175be21e-bf32-4216-9be2-f584b57516e9'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:42.691649 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-175be21e-bf32-4216-9be2-f584b57516e9 admin admin] The token cache is being invalidate because project ff8e17bfb2214ba48e9b6dd740a1dd06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:05:42.692685 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-175be21e-bf32-4216-9be2-f584b57516e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:42.694612 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-175be21e-bf32-4216-9be2-f584b57516e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:42.699691 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 721/1445] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:05:42 2022] DELETE /identity/v3/projects/ff8e17bfb2214ba48e9b6dd740a1dd06 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:42.704511 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b64664-857f-4d3e-8144-d945f3edaaa3 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:42.705042 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b64664-857f-4d3e-8144-d945f3edaaa3 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:42.705482 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b64664-857f-4d3e-8144-d945f3edaaa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:42.721627 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:42.778481 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-52b64664-857f-4d3e-8144-d945f3edaaa3 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:42.780830 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-52b64664-857f-4d3e-8144-d945f3edaaa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:42.944834 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 725/1446] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:42.953028 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05598ed-b6d2-4339-8aac-590b0fd03714 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:42.954127 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05598ed-b6d2-4339-8aac-590b0fd03714 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:42.954674 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05598ed-b6d2-4339-8aac-590b0fd03714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:42.996584 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-f05598ed-b6d2-4339-8aac-590b0fd03714 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:43.003220 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-f05598ed-b6d2-4339-8aac-590b0fd03714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:43.032389 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 722/1447] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:43.036553 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b64664-857f-4d3e-8144-d945f3edaaa3 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:43.050649 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b64664-857f-4d3e-8144-d945f3edaaa3 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:43.058660 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906a2e0-bc90-45e8-a466-510066018f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:43.059381 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906a2e0-bc90-45e8-a466-510066018f25 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:43.059730 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906a2e0-bc90-45e8-a466-510066018f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:43.060190 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906a2e0-bc90-45e8-a466-510066018f25 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:43.060972 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906a2e0-bc90-45e8-a466-510066018f25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:05:43.061350 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906a2e0-bc90-45e8-a466-510066018f25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:43.064293 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906a2e0-bc90-45e8-a466-510066018f25 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:43.080827 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 726/1448] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:05:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:05:43.084916 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05598ed-b6d2-4339-8aac-590b0fd03714 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:43.101233 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05598ed-b6d2-4339-8aac-590b0fd03714 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:43.114184 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e45d8-2f2f-42a3-9d0e-d4f0648bd27b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:43.115155 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e45d8-2f2f-42a3-9d0e-d4f0648bd27b admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:43.115648 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e45d8-2f2f-42a3-9d0e-d4f0648bd27b admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:43.116134 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672e45d8-2f2f-42a3-9d0e-d4f0648bd27b admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:43.117804 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e45d8-2f2f-42a3-9d0e-d4f0648bd27b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:43.127271 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672e45d8-2f2f-42a3-9d0e-d4f0648bd27b admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:43.135437 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:43.172514 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 723/1449] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:05:43 2022] POST /identity/v3/projects => generated 387 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:05:43.176296 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906a2e0-bc90-45e8-a466-510066018f25 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:43.189911 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906a2e0-bc90-45e8-a466-510066018f25 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:43.301620 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d970c0de-a3eb-44da-a2dd-2358b6d74de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:43.302520 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970c0de-a3eb-44da-a2dd-2358b6d74de9 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:43.302886 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970c0de-a3eb-44da-a2dd-2358b6d74de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:43.303303 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970c0de-a3eb-44da-a2dd-2358b6d74de9 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:43.304384 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d970c0de-a3eb-44da-a2dd-2358b6d74de9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:43.306906 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d970c0de-a3eb-44da-a2dd-2358b6d74de9 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:43.365481 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e45d8-2f2f-42a3-9d0e-d4f0648bd27b admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:43.368524 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 727/1450] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:05:43 2022] POST /identity/v3/users => generated 294 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:05:43.387935 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672e45d8-2f2f-42a3-9d0e-d4f0648bd27b admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:43.404109 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95983e-4b12-4e60-a021-af80d63ba3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:43.404907 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95983e-4b12-4e60-a021-af80d63ba3ae admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:43.405157 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95983e-4b12-4e60-a021-af80d63ba3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:43.405429 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95983e-4b12-4e60-a021-af80d63ba3ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:43.409507 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab95983e-4b12-4e60-a021-af80d63ba3ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:43.411513 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab95983e-4b12-4e60-a021-af80d63ba3ae admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:43.426587 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 724/1451] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 20:05:43.435125 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d970c0de-a3eb-44da-a2dd-2358b6d74de9 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:43.445329 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d970c0de-a3eb-44da-a2dd-2358b6d74de9 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:43.451113 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de34fe4-626e-45ec-95e2-6823da452914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:43.453487 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de34fe4-626e-45ec-95e2-6823da452914 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:43.453487 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de34fe4-626e-45ec-95e2-6823da452914 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:43.453487 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de34fe4-626e-45ec-95e2-6823da452914 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:43.453487 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de34fe4-626e-45ec-95e2-6823da452914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:43.455218 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de34fe4-626e-45ec-95e2-6823da452914 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:43.471752 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 728/1452] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:43 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 20:05:43.476849 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95983e-4b12-4e60-a021-af80d63ba3ae admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:43.491823 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95983e-4b12-4e60-a021-af80d63ba3ae admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:43.500815 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949097f7-fb20-4574-b2eb-dab596ac2b3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:43.501615 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949097f7-fb20-4574-b2eb-dab596ac2b3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:43.501615 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949097f7-fb20-4574-b2eb-dab596ac2b3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:43.501742 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949097f7-fb20-4574-b2eb-dab596ac2b3b admin admin] PATH_INFO: `/v3/projects/5e568b91bfec4b88ae737f7dce08b594/users/30e5f2276e6c40c888f585f90f4dbacd/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:43.539769 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949097f7-fb20-4574-b2eb-dab596ac2b3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e568b91bfec4b88ae737f7dce08b594, user_id=30e5f2276e6c40c888f585f90f4dbacd, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:43.542679 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949097f7-fb20-4574-b2eb-dab596ac2b3b admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:43.549891 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469cc42-8158-4c57-8d3a-f5fc862386e6 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:43.550137 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469cc42-8158-4c57-8d3a-f5fc862386e6 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:43.550447 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469cc42-8158-4c57-8d3a-f5fc862386e6 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:43.552746 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 729/1453] 10.222.0.13 () {54 vars in 921 bytes} [Sun Oct 23 20:05:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:05:43.593645 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 725/1454] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:43 2022] PUT /identity/v3/projects/5e568b91bfec4b88ae737f7dce08b594/users/30e5f2276e6c40c888f585f90f4dbacd/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:43.595075 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6073e1-94cb-4309-a83b-73c31e2fb303 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:43.596591 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6073e1-94cb-4309-a83b-73c31e2fb303 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:43.596591 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6073e1-94cb-4309-a83b-73c31e2fb303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:43.603295 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09564426-e394-4086-bb8c-b75554a0f2d1 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:43.603620 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09564426-e394-4086-bb8c-b75554a0f2d1 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:43.603891 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09564426-e394-4086-bb8c-b75554a0f2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:43.666418 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-09564426-e394-4086-bb8c-b75554a0f2d1 None None] MFA Rules not processed for user `312489c49a304ca9aacebd9843a4b65f`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:43.667593 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-09564426-e394-4086-bb8c-b75554a0f2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:43.675004 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-3d6073e1-94cb-4309-a83b-73c31e2fb303 None None] MFA Rules not processed for user `30e5f2276e6c40c888f585f90f4dbacd`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:43.690605 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3d6073e1-94cb-4309-a83b-73c31e2fb303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:43.837595 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:43.847694 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 730/1455] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:43.863271 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6073e1-94cb-4309-a83b-73c31e2fb303 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:43.888343 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 726/1456] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Oct 23 20:05:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:43.935889 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09564426-e394-4086-bb8c-b75554a0f2d1 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:43.937614 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-09564426-e394-4086-bb8c-b75554a0f2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:43.959981 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09564426-e394-4086-bb8c-b75554a0f2d1 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:43.984793 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e360f-1a10-45f0-b89b-c639fafb692a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312489c49a304ca9aacebd9843a4b65f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:43.985382 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e360f-1a10-45f0-b89b-c639fafb692a service placement] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:43.985797 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e360f-1a10-45f0-b89b-c639fafb692a service placement] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:43.986106 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81e360f-1a10-45f0-b89b-c639fafb692a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:44.014808 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e360f-1a10-45f0-b89b-c639fafb692a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:44.016700 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81e360f-1a10-45f0-b89b-c639fafb692a service placement] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:44.049418 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 727/1457] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 20:05:44.090277 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6073e1-94cb-4309-a83b-73c31e2fb303 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:44.104029 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110cd847-7bc0-45d6-bf01-d2040218e2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:44.104730 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110cd847-7bc0-45d6-bf01-d2040218e2b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:44.105177 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110cd847-7bc0-45d6-bf01-d2040218e2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:44.105466 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110cd847-7bc0-45d6-bf01-d2040218e2b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:44.106118 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110cd847-7bc0-45d6-bf01-d2040218e2b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:44.108286 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110cd847-7bc0-45d6-bf01-d2040218e2b8 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:44.153759 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 731/1458] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:05:43 2022] POST /identity/v3/projects => generated 385 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:44.154993 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e360f-1a10-45f0-b89b-c639fafb692a service placement] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:44.171804 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81e360f-1a10-45f0-b89b-c639fafb692a service placement] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:44.257595 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fb4ef-a819-49b1-8791-502f9bee64bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:44.258348 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fb4ef-a819-49b1-8791-502f9bee64bd admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:44.258762 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fb4ef-a819-49b1-8791-502f9bee64bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:44.259043 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fb4ef-a819-49b1-8791-502f9bee64bd admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:44.259711 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292fb4ef-a819-49b1-8791-502f9bee64bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:44.261645 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292fb4ef-a819-49b1-8791-502f9bee64bd admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:44.288890 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4497d6e-3409-45b4-ae0d-ecec21c3df3a None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:44.289301 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4497d6e-3409-45b4-ae0d-ecec21c3df3a None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:44.289517 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4497d6e-3409-45b4-ae0d-ecec21c3df3a None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:44.291106 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 732/1460] 10.222.0.13 () {54 vars in 921 bytes} [Sun Oct 23 20:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:05:44.295272 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 728/1460] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:05:44 2022] POST /identity/v3/users => generated 299 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:44.303466 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4497d6e-3409-45b4-ae0d-ecec21c3df3a None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:44.315630 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4831934-c37a-4cb2-9578-cd1e38479a69 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:44.316533 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4831934-c37a-4cb2-9578-cd1e38479a69 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:44.316818 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4831934-c37a-4cb2-9578-cd1e38479a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:44.324323 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4497d6e-3409-45b4-ae0d-ecec21c3df3a None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:44.332800 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1173acc-90a4-4579-874a-62c820002c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:44.334430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1173acc-90a4-4579-874a-62c820002c2a admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:44.334754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1173acc-90a4-4579-874a-62c820002c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:44.335659 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1173acc-90a4-4579-874a-62c820002c2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:44.336995 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1173acc-90a4-4579-874a-62c820002c2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:44.339384 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1173acc-90a4-4579-874a-62c820002c2a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:44.352257 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-f4831934-c37a-4cb2-9578-cd1e38479a69 None None] MFA Rules not processed for user `0b3d512304254ffcaa8dad90abf1f9c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:44.353571 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-f4831934-c37a-4cb2-9578-cd1e38479a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:44.354469 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 733/1461] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:44.356671 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1173acc-90a4-4579-874a-62c820002c2a admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:44.368683 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1173acc-90a4-4579-874a-62c820002c2a admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:44.374190 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf36e6-2bec-468e-a46c-1836afd192aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:44.375374 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf36e6-2bec-468e-a46c-1836afd192aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:44.375374 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf36e6-2bec-468e-a46c-1836afd192aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:44.375544 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bf36e6-2bec-468e-a46c-1836afd192aa admin admin] PATH_INFO: `/v3/projects/5b7ef5f9334544c79df0bc69a4f4afc8/users/47734a57223940ea9141915df6aa4abe/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:44.378443 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:44.415325 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf36e6-2bec-468e-a46c-1836afd192aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b7ef5f9334544c79df0bc69a4f4afc8, user_id=47734a57223940ea9141915df6aa4abe, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:44.423988 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bf36e6-2bec-468e-a46c-1836afd192aa admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:44.450070 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 734/1462] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:44 2022] PUT /identity/v3/projects/5b7ef5f9334544c79df0bc69a4f4afc8/users/47734a57223940ea9141915df6aa4abe/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:05:44.450935 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 729/1463] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Oct 23 20:05:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:44.453783 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf36e6-2bec-468e-a46c-1836afd192aa admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:44.469367 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4831934-c37a-4cb2-9578-cd1e38479a69 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:44.471749 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-f4831934-c37a-4cb2-9578-cd1e38479a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:44.498554 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4831934-c37a-4cb2-9578-cd1e38479a69 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:44.557644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bf36e6-2bec-468e-a46c-1836afd192aa admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:44.563843 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae79723-d2f7-4e05-b52b-873e391bf0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:44.564777 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae79723-d2f7-4e05-b52b-873e391bf0b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:44.565123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae79723-d2f7-4e05-b52b-873e391bf0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:44.565470 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae79723-d2f7-4e05-b52b-873e391bf0b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:44.566184 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae79723-d2f7-4e05-b52b-873e391bf0b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:44.568034 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae79723-d2f7-4e05-b52b-873e391bf0b8 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:44.583058 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 735/1464] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 20:05:44.587146 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae79723-d2f7-4e05-b52b-873e391bf0b8 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:44.593897 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763ecc89-4b9b-4b71-a386-f9ed5a2afe18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:44.594187 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ecc89-4b9b-4b71-a386-f9ed5a2afe18 service nova] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:44.594588 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ecc89-4b9b-4b71-a386-f9ed5a2afe18 service nova] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:44.594927 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763ecc89-4b9b-4b71-a386-f9ed5a2afe18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:44.597676 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae79723-d2f7-4e05-b52b-873e391bf0b8 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:44.602913 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be2fa3-25eb-46e0-a728-a959ba1d18a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:44.603583 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be2fa3-25eb-46e0-a728-a959ba1d18a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:44.603868 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be2fa3-25eb-46e0-a728-a959ba1d18a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:44.604144 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83be2fa3-25eb-46e0-a728-a959ba1d18a6 admin admin] PATH_INFO: `/v3/projects/5b7ef5f9334544c79df0bc69a4f4afc8/users/47734a57223940ea9141915df6aa4abe/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:44.606278 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be2fa3-25eb-46e0-a728-a959ba1d18a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b7ef5f9334544c79df0bc69a4f4afc8, user_id=47734a57223940ea9141915df6aa4abe, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:44.606828 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763ecc89-4b9b-4b71-a386-f9ed5a2afe18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:44.608757 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763ecc89-4b9b-4b71-a386-f9ed5a2afe18 service nova] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:44.609073 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83be2fa3-25eb-46e0-a728-a959ba1d18a6 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:44.621132 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 730/1465] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 20:05:44.624005 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 736/1466] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:44 2022] PUT /identity/v3/projects/5b7ef5f9334544c79df0bc69a4f4afc8/users/47734a57223940ea9141915df6aa4abe/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:44.636149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6388c9d-7d31-465e-8b3c-3cbae9245748 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:44.636599 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6388c9d-7d31-465e-8b3c-3cbae9245748 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:44.636881 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6388c9d-7d31-465e-8b3c-3cbae9245748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:44.672289 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-c6388c9d-7d31-465e-8b3c-3cbae9245748 None None] MFA Rules not processed for user `47734a57223940ea9141915df6aa4abe`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:44.673441 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-c6388c9d-7d31-465e-8b3c-3cbae9245748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:44.743145 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 731/1467] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:44 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:05:44.754113 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be2fa3-25eb-46e0-a728-a959ba1d18a6 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:44.756555 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-83be2fa3-25eb-46e0-a728-a959ba1d18a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:44.798294 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:44.823981 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83be2fa3-25eb-46e0-a728-a959ba1d18a6 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:44.827808 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed2083-8d69-4a44-bfec-06f3af44f796 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:44.828491 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed2083-8d69-4a44-bfec-06f3af44f796 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:44.828841 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed2083-8d69-4a44-bfec-06f3af44f796 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:44.829182 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed2083-8d69-4a44-bfec-06f3af44f796 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:44.838603 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ed2083-8d69-4a44-bfec-06f3af44f796 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:44.840214 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ed2083-8d69-4a44-bfec-06f3af44f796 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:44.849786 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 737/1468] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:44 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:05:45.075308 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6388c9d-7d31-465e-8b3c-3cbae9245748 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:45.076833 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-c6388c9d-7d31-465e-8b3c-3cbae9245748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:45.133307 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:45.185134 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6388c9d-7d31-465e-8b3c-3cbae9245748 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:45.190370 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d7f07-036a-482a-88cc-c5cf477bbac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:45.191020 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d7f07-036a-482a-88cc-c5cf477bbac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:45.191296 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d7f07-036a-482a-88cc-c5cf477bbac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:45.191596 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6d7f07-036a-482a-88cc-c5cf477bbac1 admin admin] PATH_INFO: `/v3/users/a5a9fe7502cf4f1d8a4c6406e2ff77f8` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:45.192740 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6d7f07-036a-482a-88cc-c5cf477bbac1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5a9fe7502cf4f1d8a4c6406e2ff77f8)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:45.195410 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6d7f07-036a-482a-88cc-c5cf477bbac1 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:45.249970 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ba6d7f07-036a-482a-88cc-c5cf477bbac1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5a9fe7502cf4f1d8a4c6406e2ff77f8'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:45.264701 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ba6d7f07-036a-482a-88cc-c5cf477bbac1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5a9fe7502cf4f1d8a4c6406e2ff77f8'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:45.275065 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ba6d7f07-036a-482a-88cc-c5cf477bbac1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5a9fe7502cf4f1d8a4c6406e2ff77f8'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:45.284820 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ba6d7f07-036a-482a-88cc-c5cf477bbac1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5a9fe7502cf4f1d8a4c6406e2ff77f8'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:45.285630 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ba6d7f07-036a-482a-88cc-c5cf477bbac1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5a9fe7502cf4f1d8a4c6406e2ff77f8'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:45.297940 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 732/1469] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:05:45 2022] DELETE /identity/v3/users/a5a9fe7502cf4f1d8a4c6406e2ff77f8 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:45.576736 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed2083-8d69-4a44-bfec-06f3af44f796 service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:45.615651 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed2083-8d69-4a44-bfec-06f3af44f796 service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:45.752540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2746e-30cc-4037-853e-503d28656884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:45.753199 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2746e-30cc-4037-853e-503d28656884 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:45.753475 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2746e-30cc-4037-853e-503d28656884 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:45.753749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca2746e-30cc-4037-853e-503d28656884 admin admin] PATH_INFO: `/v3/projects/42ceda910739400390f5074a7c8f0e9d` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:45.754937 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca2746e-30cc-4037-853e-503d28656884 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42ceda910739400390f5074a7c8f0e9d)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:45.756993 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca2746e-30cc-4037-853e-503d28656884 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:45.768887 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d7f07-036a-482a-88cc-c5cf477bbac1 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:45.775139 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-ba6d7f07-036a-482a-88cc-c5cf477bbac1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:45.795724 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6d7f07-036a-482a-88cc-c5cf477bbac1 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:45.802806 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69997e5-483f-4160-a163-f1cd4ab2e11a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:45.803610 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69997e5-483f-4160-a163-f1cd4ab2e11a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:45.804029 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69997e5-483f-4160-a163-f1cd4ab2e11a admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:45.804403 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69997e5-483f-4160-a163-f1cd4ab2e11a admin admin] PATH_INFO: `/v3/users/30e5f2276e6c40c888f585f90f4dbacd` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:45.806041 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69997e5-483f-4160-a163-f1cd4ab2e11a admin admin] RBAC: Authorizing `identity:delete_user(user_id=30e5f2276e6c40c888f585f90f4dbacd)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:45.808850 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69997e5-483f-4160-a163-f1cd4ab2e11a admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:45.870700 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4ca2746e-30cc-4037-853e-503d28656884 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42ceda910739400390f5074a7c8f0e9d', 'request_id': 'req-4ca2746e-30cc-4037-853e-503d28656884'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:45.895262 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4ca2746e-30cc-4037-853e-503d28656884 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42ceda910739400390f5074a7c8f0e9d', 'request_id': 'req-4ca2746e-30cc-4037-853e-503d28656884'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:45.904703 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4ca2746e-30cc-4037-853e-503d28656884 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42ceda910739400390f5074a7c8f0e9d', 'request_id': 'req-4ca2746e-30cc-4037-853e-503d28656884'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:45.920464 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-d69997e5-483f-4160-a163-f1cd4ab2e11a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30e5f2276e6c40c888f585f90f4dbacd'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:45.934774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4ca2746e-30cc-4037-853e-503d28656884 admin admin] The token cache is being invalidate because project 42ceda910739400390f5074a7c8f0e9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:05:45.937624 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4ca2746e-30cc-4037-853e-503d28656884 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:45.939717 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4ca2746e-30cc-4037-853e-503d28656884 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:45.942773 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 738/1470] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:05:45 2022] DELETE /identity/v3/projects/42ceda910739400390f5074a7c8f0e9d => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:05:45.948231 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2746e-30cc-4037-853e-503d28656884 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:45.950386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4ca2746e-30cc-4037-853e-503d28656884 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:45.955191 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-d69997e5-483f-4160-a163-f1cd4ab2e11a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30e5f2276e6c40c888f585f90f4dbacd'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:45.972464 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-d69997e5-483f-4160-a163-f1cd4ab2e11a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30e5f2276e6c40c888f585f90f4dbacd'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:45.987926 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-d69997e5-483f-4160-a163-f1cd4ab2e11a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30e5f2276e6c40c888f585f90f4dbacd'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:45.989042 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-d69997e5-483f-4160-a163-f1cd4ab2e11a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30e5f2276e6c40c888f585f90f4dbacd'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:46.012921 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 733/1471] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:05:45 2022] DELETE /identity/v3/users/30e5f2276e6c40c888f585f90f4dbacd => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:46.015358 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ee656-7ae9-4c83-8d57-4ef3d2acfb41 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:46.018409 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ee656-7ae9-4c83-8d57-4ef3d2acfb41 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:46.019543 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8ee656-7ae9-4c83-8d57-4ef3d2acfb41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:46.061563 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-0e8ee656-7ae9-4c83-8d57-4ef3d2acfb41 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:46.062873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-0e8ee656-7ae9-4c83-8d57-4ef3d2acfb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:46.083702 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:46.158417 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca2746e-30cc-4037-853e-503d28656884 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:46.167820 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 734/1472] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:46.175420 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9251c-f79e-4cd1-9f39-ec6838bbf0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:46.176089 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9251c-f79e-4cd1-9f39-ec6838bbf0d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:46.176370 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9251c-f79e-4cd1-9f39-ec6838bbf0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:46.176649 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9251c-f79e-4cd1-9f39-ec6838bbf0d1 admin admin] PATH_INFO: `/v3/users/ad67526a2e6d4d3bba96fc2f6405b756` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:46.178065 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b9251c-f79e-4cd1-9f39-ec6838bbf0d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad67526a2e6d4d3bba96fc2f6405b756)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:46.180238 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b9251c-f79e-4cd1-9f39-ec6838bbf0d1 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:46.180589 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ee656-7ae9-4c83-8d57-4ef3d2acfb41 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:46.187184 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-0e8ee656-7ae9-4c83-8d57-4ef3d2acfb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:46.206983 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:46.322459 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-11b9251c-f79e-4cd1-9f39-ec6838bbf0d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad67526a2e6d4d3bba96fc2f6405b756'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:46.340654 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8ee656-7ae9-4c83-8d57-4ef3d2acfb41 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:46.347106 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c6f535-9be6-4a18-8b35-cce201988145 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:46.347106 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6f535-9be6-4a18-8b35-cce201988145 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:46.347106 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6f535-9be6-4a18-8b35-cce201988145 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:46.347106 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c6f535-9be6-4a18-8b35-cce201988145 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:46.357969 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-11b9251c-f79e-4cd1-9f39-ec6838bbf0d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad67526a2e6d4d3bba96fc2f6405b756'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:46.370145 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c6f535-9be6-4a18-8b35-cce201988145 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:46.371737 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-11b9251c-f79e-4cd1-9f39-ec6838bbf0d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad67526a2e6d4d3bba96fc2f6405b756'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:46.372652 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-11b9251c-f79e-4cd1-9f39-ec6838bbf0d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad67526a2e6d4d3bba96fc2f6405b756'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:46.374161 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c6f535-9be6-4a18-8b35-cce201988145 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:46.388506 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-11b9251c-f79e-4cd1-9f39-ec6838bbf0d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad67526a2e6d4d3bba96fc2f6405b756'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:46.399995 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 739/1474] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:05:45 2022] DELETE /identity/v3/users/ad67526a2e6d4d3bba96fc2f6405b756 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:46.414360 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 735/1474] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:05:46.851547 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9251c-f79e-4cd1-9f39-ec6838bbf0d1 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:46.853084 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-11b9251c-f79e-4cd1-9f39-ec6838bbf0d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:46.931757 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9251c-f79e-4cd1-9f39-ec6838bbf0d1 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:46.938795 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ecd367-a2bf-4fd0-b085-989653170a9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:46.939666 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecd367-a2bf-4fd0-b085-989653170a9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:46.940028 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecd367-a2bf-4fd0-b085-989653170a9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:46.940232 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecd367-a2bf-4fd0-b085-989653170a9a admin admin] PATH_INFO: `/v3/projects/9086a333071641999c47ccbeb4df233f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:46.942044 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ecd367-a2bf-4fd0-b085-989653170a9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9086a333071641999c47ccbeb4df233f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:46.944438 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ecd367-a2bf-4fd0-b085-989653170a9a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:47.035972 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e1ecd367-a2bf-4fd0-b085-989653170a9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9086a333071641999c47ccbeb4df233f', 'request_id': 'req-e1ecd367-a2bf-4fd0-b085-989653170a9a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:47.046963 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c6f535-9be6-4a18-8b35-cce201988145 service neutron] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:47.049068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-70c6f535-9be6-4a18-8b35-cce201988145 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:47.051438 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e1ecd367-a2bf-4fd0-b085-989653170a9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9086a333071641999c47ccbeb4df233f', 'request_id': 'req-e1ecd367-a2bf-4fd0-b085-989653170a9a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:47.067873 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e1ecd367-a2bf-4fd0-b085-989653170a9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9086a333071641999c47ccbeb4df233f', 'request_id': 'req-e1ecd367-a2bf-4fd0-b085-989653170a9a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:47.068100 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c6f535-9be6-4a18-8b35-cce201988145 service neutron] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:47.107346 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e1ecd367-a2bf-4fd0-b085-989653170a9a admin admin] The token cache is being invalidate because project 9086a333071641999c47ccbeb4df233f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:05:47.107665 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e1ecd367-a2bf-4fd0-b085-989653170a9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:47.108340 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e1ecd367-a2bf-4fd0-b085-989653170a9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:47.109880 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 740/1475] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:05:46 2022] DELETE /identity/v3/projects/9086a333071641999c47ccbeb4df233f => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:47.115195 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c47a7-0139-489a-8db0-268ef659a836 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:47.115562 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c47a7-0139-489a-8db0-268ef659a836 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:47.115917 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c47a7-0139-489a-8db0-268ef659a836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:47.137160 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-297c47a7-0139-489a-8db0-268ef659a836 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:47.138530 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-297c47a7-0139-489a-8db0-268ef659a836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:47.149446 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:47.157098 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c701a-b6a1-4fbe-adea-8b44c98b1b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:47.157806 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c701a-b6a1-4fbe-adea-8b44c98b1b49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:47.158096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c701a-b6a1-4fbe-adea-8b44c98b1b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:47.158411 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c701a-b6a1-4fbe-adea-8b44c98b1b49 admin admin] PATH_INFO: `/v3/projects/5e568b91bfec4b88ae737f7dce08b594` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:47.159552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769c701a-b6a1-4fbe-adea-8b44c98b1b49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e568b91bfec4b88ae737f7dce08b594)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:47.161469 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769c701a-b6a1-4fbe-adea-8b44c98b1b49 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:47.208206 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 741/1476] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:47.211439 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-769c701a-b6a1-4fbe-adea-8b44c98b1b49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e568b91bfec4b88ae737f7dce08b594', 'request_id': 'req-769c701a-b6a1-4fbe-adea-8b44c98b1b49'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:47.215809 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cdf363-0718-4d65-91cd-83211ce537a0 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:47.216105 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cdf363-0718-4d65-91cd-83211ce537a0 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:47.216452 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cdf363-0718-4d65-91cd-83211ce537a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:47.220955 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-769c701a-b6a1-4fbe-adea-8b44c98b1b49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e568b91bfec4b88ae737f7dce08b594', 'request_id': 'req-769c701a-b6a1-4fbe-adea-8b44c98b1b49'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:47.227233 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:47.229890 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-769c701a-b6a1-4fbe-adea-8b44c98b1b49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e568b91bfec4b88ae737f7dce08b594', 'request_id': 'req-769c701a-b6a1-4fbe-adea-8b44c98b1b49'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:47.240514 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-e4cdf363-0718-4d65-91cd-83211ce537a0 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:47.242469 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e4cdf363-0718-4d65-91cd-83211ce537a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:47.246060 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-769c701a-b6a1-4fbe-adea-8b44c98b1b49 admin admin] The token cache is being invalidate because project 5e568b91bfec4b88ae737f7dce08b594 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:05:47.246513 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-769c701a-b6a1-4fbe-adea-8b44c98b1b49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:47.247370 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-769c701a-b6a1-4fbe-adea-8b44c98b1b49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:47.250092 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 736/1477] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:05:47 2022] DELETE /identity/v3/projects/5e568b91bfec4b88ae737f7dce08b594 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:47.254399 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c701a-b6a1-4fbe-adea-8b44c98b1b49 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:47.255729 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-769c701a-b6a1-4fbe-adea-8b44c98b1b49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:47.314379 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 742/1478] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:47.315324 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cdf363-0718-4d65-91cd-83211ce537a0 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:47.326177 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4cdf363-0718-4d65-91cd-83211ce537a0 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:47.331921 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c45dae0-2851-4a99-a1d3-7041f078d138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:47.332745 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45dae0-2851-4a99-a1d3-7041f078d138 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:47.333032 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45dae0-2851-4a99-a1d3-7041f078d138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:47.333310 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45dae0-2851-4a99-a1d3-7041f078d138 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:47.334494 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c45dae0-2851-4a99-a1d3-7041f078d138 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:05:47.334810 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c45dae0-2851-4a99-a1d3-7041f078d138 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:47.336558 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c701a-b6a1-4fbe-adea-8b44c98b1b49 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:47.336920 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c45dae0-2851-4a99-a1d3-7041f078d138 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:47.341826 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea078820-7b02-462b-b404-9c045e3e88e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:47.342512 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea078820-7b02-462b-b404-9c045e3e88e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:47.342794 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea078820-7b02-462b-b404-9c045e3e88e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:47.343068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea078820-7b02-462b-b404-9c045e3e88e1 admin admin] PATH_INFO: `/v3/users/47734a57223940ea9141915df6aa4abe` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:47.344222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea078820-7b02-462b-b404-9c045e3e88e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47734a57223940ea9141915df6aa4abe)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:47.346149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea078820-7b02-462b-b404-9c045e3e88e1 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:47.358711 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 743/1479] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:05:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:05:47.362970 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c45dae0-2851-4a99-a1d3-7041f078d138 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:47.373067 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c45dae0-2851-4a99-a1d3-7041f078d138 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:47.379872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ee7d2-ccc3-430e-8b9b-d377d62ff4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:47.380484 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ee7d2-ccc3-430e-8b9b-d377d62ff4b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:47.380767 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ee7d2-ccc3-430e-8b9b-d377d62ff4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:47.381046 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ee7d2-ccc3-430e-8b9b-d377d62ff4b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:47.381798 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ee7d2-ccc3-430e-8b9b-d377d62ff4b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:47.384189 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ee7d2-ccc3-430e-8b9b-d377d62ff4b4 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:47.398437 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ea078820-7b02-462b-b404-9c045e3e88e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47734a57223940ea9141915df6aa4abe'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:47.410480 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ea078820-7b02-462b-b404-9c045e3e88e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47734a57223940ea9141915df6aa4abe'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:47.412782 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 744/1480] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:05:47 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:05:47.417413 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ee7d2-ccc3-430e-8b9b-d377d62ff4b4 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:47.420451 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ea078820-7b02-462b-b404-9c045e3e88e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47734a57223940ea9141915df6aa4abe'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:47.429037 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ea078820-7b02-462b-b404-9c045e3e88e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47734a57223940ea9141915df6aa4abe'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:47.429839 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ea078820-7b02-462b-b404-9c045e3e88e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47734a57223940ea9141915df6aa4abe'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:47.436213 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 737/1481] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:05:47 2022] DELETE /identity/v3/users/47734a57223940ea9141915df6aa4abe => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:05:47.482205 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ee7d2-ccc3-430e-8b9b-d377d62ff4b4 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:47.510203 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:47.542988 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8310d19a-4b2f-40b8-bf93-ec8854b90691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:47.543618 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8310d19a-4b2f-40b8-bf93-ec8854b90691 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:47.543918 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8310d19a-4b2f-40b8-bf93-ec8854b90691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:47.544184 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8310d19a-4b2f-40b8-bf93-ec8854b90691 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:47.544825 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8310d19a-4b2f-40b8-bf93-ec8854b90691 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:47.546809 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8310d19a-4b2f-40b8-bf93-ec8854b90691 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:47.569857 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 745/1482] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:05:47 2022] POST /identity/v3/users => generated 294 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:47.577699 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea078820-7b02-462b-b404-9c045e3e88e1 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:47.579159 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-ea078820-7b02-462b-b404-9c045e3e88e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:47.593168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea078820-7b02-462b-b404-9c045e3e88e1 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:47.597923 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484e287-6287-4b68-9d7e-f7d25649b160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:47.598565 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484e287-6287-4b68-9d7e-f7d25649b160 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:47.598847 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484e287-6287-4b68-9d7e-f7d25649b160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:47.599121 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2484e287-6287-4b68-9d7e-f7d25649b160 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:47.599791 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2484e287-6287-4b68-9d7e-f7d25649b160 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:47.601962 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2484e287-6287-4b68-9d7e-f7d25649b160 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:47.612168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 738/1483] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:47.616065 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8310d19a-4b2f-40b8-bf93-ec8854b90691 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:47.626452 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8310d19a-4b2f-40b8-bf93-ec8854b90691 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:47.631508 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf6f29-bb2f-4c6b-90e5-8aec6f0d2847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:47.631732 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf6f29-bb2f-4c6b-90e5-8aec6f0d2847 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:47.632540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf6f29-bb2f-4c6b-90e5-8aec6f0d2847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:47.633005 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf6f29-bb2f-4c6b-90e5-8aec6f0d2847 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:47.633665 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bf6f29-bb2f-4c6b-90e5-8aec6f0d2847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:47.635294 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8bf6f29-bb2f-4c6b-90e5-8aec6f0d2847 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:47.645548 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 746/1484] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:47.649836 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484e287-6287-4b68-9d7e-f7d25649b160 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:47.659855 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2484e287-6287-4b68-9d7e-f7d25649b160 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:47.665072 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c60ae6-6fec-40c2-badf-f8ff74a73b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:47.665729 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c60ae6-6fec-40c2-badf-f8ff74a73b18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:47.666007 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c60ae6-6fec-40c2-badf-f8ff74a73b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:47.666310 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c60ae6-6fec-40c2-badf-f8ff74a73b18 admin admin] PATH_INFO: `/v3/projects/0b6f1dab6a5c4285a9bfaa9657a9e40a/users/4a202356df2f48dc870c18835ea07034/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:47.682354 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c60ae6-6fec-40c2-badf-f8ff74a73b18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b6f1dab6a5c4285a9bfaa9657a9e40a, user_id=4a202356df2f48dc870c18835ea07034, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:47.685175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c60ae6-6fec-40c2-badf-f8ff74a73b18 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:47.700344 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 739/1485] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:47 2022] PUT /identity/v3/projects/0b6f1dab6a5c4285a9bfaa9657a9e40a/users/4a202356df2f48dc870c18835ea07034/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:47.713508 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9030ba-8bb0-4f76-9b8b-9e1eb3e0a02f None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:47.714175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c60ae6-6fec-40c2-badf-f8ff74a73b18 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:47.714362 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9030ba-8bb0-4f76-9b8b-9e1eb3e0a02f None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:47.714782 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9030ba-8bb0-4f76-9b8b-9e1eb3e0a02f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:47.716261 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-46c60ae6-6fec-40c2-badf-f8ff74a73b18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:47.750614 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-6c9030ba-8bb0-4f76-9b8b-9e1eb3e0a02f None None] MFA Rules not processed for user `4a202356df2f48dc870c18835ea07034`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:47.752038 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-6c9030ba-8bb0-4f76-9b8b-9e1eb3e0a02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:47.770991 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:47.807973 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c60ae6-6fec-40c2-badf-f8ff74a73b18 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:47.813531 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a11b79d-e74a-4d0c-a3ca-f0897864f10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:47.815550 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11b79d-e74a-4d0c-a3ca-f0897864f10f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:47.815841 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11b79d-e74a-4d0c-a3ca-f0897864f10f admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:47.816312 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11b79d-e74a-4d0c-a3ca-f0897864f10f admin admin] PATH_INFO: `/v3/projects/5b7ef5f9334544c79df0bc69a4f4afc8` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:47.817688 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a11b79d-e74a-4d0c-a3ca-f0897864f10f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b7ef5f9334544c79df0bc69a4f4afc8)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:47.820760 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a11b79d-e74a-4d0c-a3ca-f0897864f10f admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:47.837615 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 747/1486] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:47.842282 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9030ba-8bb0-4f76-9b8b-9e1eb3e0a02f None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:47.855876 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9030ba-8bb0-4f76-9b8b-9e1eb3e0a02f None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:47.863961 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6192b-ac83-42f1-96b2-2de0d0e1444d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:47.869159 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6192b-ac83-42f1-96b2-2de0d0e1444d admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:47.869457 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6192b-ac83-42f1-96b2-2de0d0e1444d admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:47.869769 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb6192b-ac83-42f1-96b2-2de0d0e1444d admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:47.870500 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb6192b-ac83-42f1-96b2-2de0d0e1444d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:47.872634 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb6192b-ac83-42f1-96b2-2de0d0e1444d admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:47.894989 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-2a11b79d-e74a-4d0c-a3ca-f0897864f10f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b7ef5f9334544c79df0bc69a4f4afc8', 'request_id': 'req-2a11b79d-e74a-4d0c-a3ca-f0897864f10f'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:47.904428 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-2a11b79d-e74a-4d0c-a3ca-f0897864f10f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b7ef5f9334544c79df0bc69a4f4afc8', 'request_id': 'req-2a11b79d-e74a-4d0c-a3ca-f0897864f10f'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:47.913163 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-2a11b79d-e74a-4d0c-a3ca-f0897864f10f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b7ef5f9334544c79df0bc69a4f4afc8', 'request_id': 'req-2a11b79d-e74a-4d0c-a3ca-f0897864f10f'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:47.913525 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 748/1487] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:05:47 2022] POST /identity/v3/projects => generated 387 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:47.914585 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6192b-ac83-42f1-96b2-2de0d0e1444d admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:47.923025 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-2a11b79d-e74a-4d0c-a3ca-f0897864f10f admin admin] The token cache is being invalidate because project 5b7ef5f9334544c79df0bc69a4f4afc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:05:47.923399 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-2a11b79d-e74a-4d0c-a3ca-f0897864f10f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:47.924231 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-2a11b79d-e74a-4d0c-a3ca-f0897864f10f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:47.925724 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 740/1488] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:05:47 2022] DELETE /identity/v3/projects/5b7ef5f9334544c79df0bc69a4f4afc8 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:47.931810 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d1172-1ef4-4685-82fa-f05657145d7a None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:47.932079 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d1172-1ef4-4685-82fa-f05657145d7a None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:47.932352 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1d1172-1ef4-4685-82fa-f05657145d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:47.952072 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-fb1d1172-1ef4-4685-82fa-f05657145d7a None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:47.953520 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-fb1d1172-1ef4-4685-82fa-f05657145d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:47.989325 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb6192b-ac83-42f1-96b2-2de0d0e1444d admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:47.995520 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b5f6b-363c-45ed-88d1-4ac2906fd267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:47.996233 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b5f6b-363c-45ed-88d1-4ac2906fd267 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:47.996586 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b5f6b-363c-45ed-88d1-4ac2906fd267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:47.996931 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1b5f6b-363c-45ed-88d1-4ac2906fd267 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:47.997670 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b5f6b-363c-45ed-88d1-4ac2906fd267 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:47.999699 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1b5f6b-363c-45ed-88d1-4ac2906fd267 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:48.039189 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 749/1489] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:05:47 2022] POST /identity/v3/users => generated 300 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:48.041833 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b5f6b-363c-45ed-88d1-4ac2906fd267 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:48.043639 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ac1b5f6b-363c-45ed-88d1-4ac2906fd267 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:48.046565 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 741/1490] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:05:48.054445 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a59110-6afc-4d21-8f6c-c747d24bea79 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:48.055127 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:48.055228 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a59110-6afc-4d21-8f6c-c747d24bea79 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:48.055228 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a59110-6afc-4d21-8f6c-c747d24bea79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:48.059190 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1b5f6b-363c-45ed-88d1-4ac2906fd267 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:48.064925 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c0f1a-a95b-4a33-b1ca-1e491c6598be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:48.065538 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c0f1a-a95b-4a33-b1ca-1e491c6598be admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:48.066116 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c0f1a-a95b-4a33-b1ca-1e491c6598be admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:48.066568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c0f1a-a95b-4a33-b1ca-1e491c6598be admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:48.067414 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2c0f1a-a95b-4a33-b1ca-1e491c6598be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:48.069652 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2c0f1a-a95b-4a33-b1ca-1e491c6598be admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:48.078405 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-70a59110-6afc-4d21-8f6c-c747d24bea79 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:48.079620 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-70a59110-6afc-4d21-8f6c-c747d24bea79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:48.086600 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 750/1491] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:48.087571 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c0f1a-a95b-4a33-b1ca-1e491c6598be admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:48.089684 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 742/1492] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:05:48.093884 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a59110-6afc-4d21-8f6c-c747d24bea79 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:48.098147 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c0f1a-a95b-4a33-b1ca-1e491c6598be admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:48.103209 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ade1e-0776-44c6-a0a3-01ce04545183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:48.103877 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ade1e-0776-44c6-a0a3-01ce04545183 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:48.104159 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ade1e-0776-44c6-a0a3-01ce04545183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:48.104558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ade1e-0776-44c6-a0a3-01ce04545183 admin admin] PATH_INFO: `/v3/projects/a084917827a64336b7537e0e4cc8c3ee/users/ce78583d1272418fb5adc985438a5ba5/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:48.105196 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a59110-6afc-4d21-8f6c-c747d24bea79 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:48.111171 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c5fc7f-32a0-4efb-8d41-d030de3ceadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:48.111775 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5fc7f-32a0-4efb-8d41-d030de3ceadb admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:48.112054 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5fc7f-32a0-4efb-8d41-d030de3ceadb admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:48.112319 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c5fc7f-32a0-4efb-8d41-d030de3ceadb admin admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:48.112965 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c5fc7f-32a0-4efb-8d41-d030de3ceadb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:05:48.113269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c5fc7f-32a0-4efb-8d41-d030de3ceadb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:48.116903 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c5fc7f-32a0-4efb-8d41-d030de3ceadb admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:48.132743 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 743/1493] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:05:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:05:48.135213 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c5fc7f-32a0-4efb-8d41-d030de3ceadb admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:48.136711 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03ade1e-0776-44c6-a0a3-01ce04545183 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a084917827a64336b7537e0e4cc8c3ee, user_id=ce78583d1272418fb5adc985438a5ba5, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:48.140402 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03ade1e-0776-44c6-a0a3-01ce04545183 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:48.145323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c5fc7f-32a0-4efb-8d41-d030de3ceadb admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:48.150622 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473d574-9684-4361-a89e-d6e1e28f0483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:48.151226 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473d574-9684-4361-a89e-d6e1e28f0483 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:48.151506 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473d574-9684-4361-a89e-d6e1e28f0483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:48.151781 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9473d574-9684-4361-a89e-d6e1e28f0483 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:48.152408 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9473d574-9684-4361-a89e-d6e1e28f0483 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:48.154303 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9473d574-9684-4361-a89e-d6e1e28f0483 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:48.164872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 751/1494] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:48 2022] PUT /identity/v3/projects/a084917827a64336b7537e0e4cc8c3ee/users/ce78583d1272418fb5adc985438a5ba5/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:48.168794 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ade1e-0776-44c6-a0a3-01ce04545183 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:48.184650 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 744/1495] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:05:48 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:48.195366 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473d574-9684-4361-a89e-d6e1e28f0483 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:48.227238 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:48.236814 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03ade1e-0776-44c6-a0a3-01ce04545183 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:48.269874 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9473d574-9684-4361-a89e-d6e1e28f0483 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:48.275910 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e7c29a-a36e-4755-aa32-0ab7ee626e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:48.276520 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e7c29a-a36e-4755-aa32-0ab7ee626e9c admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:48.276792 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e7c29a-a36e-4755-aa32-0ab7ee626e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:48.277058 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e7c29a-a36e-4755-aa32-0ab7ee626e9c admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:48.277694 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e7c29a-a36e-4755-aa32-0ab7ee626e9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:48.280159 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e7c29a-a36e-4755-aa32-0ab7ee626e9c admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:48.301014 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f5dbdb-953c-47d3-a7f8-9f36e1ed324d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:48.301484 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5dbdb-953c-47d3-a7f8-9f36e1ed324d admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:48.301773 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5dbdb-953c-47d3-a7f8-9f36e1ed324d admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:48.302059 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5dbdb-953c-47d3-a7f8-9f36e1ed324d admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:48.302981 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f5dbdb-953c-47d3-a7f8-9f36e1ed324d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:48.304516 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f5dbdb-953c-47d3-a7f8-9f36e1ed324d admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:48.309421 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 745/1496] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:05:48 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:48.310073 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e7c29a-a36e-4755-aa32-0ab7ee626e9c admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:48.322414 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 752/1497] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 20:05:48.322557 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e7c29a-a36e-4755-aa32-0ab7ee626e9c admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:48.323566 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f5dbdb-953c-47d3-a7f8-9f36e1ed324d admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:48.325832 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34b0ce-ca06-4ed8-a759-3564197fa1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:48.326464 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34b0ce-ca06-4ed8-a759-3564197fa1d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:48.326754 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34b0ce-ca06-4ed8-a759-3564197fa1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:48.327162 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f34b0ce-ca06-4ed8-a759-3564197fa1d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:48.327762 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34b0ce-ca06-4ed8-a759-3564197fa1d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:48.329526 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f34b0ce-ca06-4ed8-a759-3564197fa1d0 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:48.333374 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f5dbdb-953c-47d3-a7f8-9f36e1ed324d admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:48.338201 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6ab43-3528-4a5a-bd56-1c4b544f62b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:48.338955 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6ab43-3528-4a5a-bd56-1c4b544f62b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:48.339237 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6ab43-3528-4a5a-bd56-1c4b544f62b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:48.339510 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6ab43-3528-4a5a-bd56-1c4b544f62b1 admin admin] PATH_INFO: `/v3/projects/a084917827a64336b7537e0e4cc8c3ee/users/ce78583d1272418fb5adc985438a5ba5/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:48.339968 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 746/1498] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:48.341517 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc6ab43-3528-4a5a-bd56-1c4b544f62b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a084917827a64336b7537e0e4cc8c3ee, user_id=ce78583d1272418fb5adc985438a5ba5, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:48.343539 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34b0ce-ca06-4ed8-a759-3564197fa1d0 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:48.344206 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc6ab43-3528-4a5a-bd56-1c4b544f62b1 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:48.354019 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f34b0ce-ca06-4ed8-a759-3564197fa1d0 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:48.355062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 753/1499] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:48 2022] PUT /identity/v3/projects/a084917827a64336b7537e0e4cc8c3ee/users/ce78583d1272418fb5adc985438a5ba5/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:48.364963 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a429690-98db-448b-8ce5-0b0fa21e0d35 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:48.365261 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a429690-98db-448b-8ce5-0b0fa21e0d35 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:48.365539 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a429690-98db-448b-8ce5-0b0fa21e0d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:48.395299 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-9a429690-98db-448b-8ce5-0b0fa21e0d35 None None] MFA Rules not processed for user `ce78583d1272418fb5adc985438a5ba5`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:48.396557 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9a429690-98db-448b-8ce5-0b0fa21e0d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:48.421309 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31631d3c-f84c-444b-8147-be50a00c15fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:48.421920 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31631d3c-f84c-444b-8147-be50a00c15fc admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:48.422200 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31631d3c-f84c-444b-8147-be50a00c15fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:48.422505 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31631d3c-f84c-444b-8147-be50a00c15fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:48.423152 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31631d3c-f84c-444b-8147-be50a00c15fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:48.424892 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31631d3c-f84c-444b-8147-be50a00c15fc admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:48.437122 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 747/1500] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:48 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 20:05:48.441135 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31631d3c-f84c-444b-8147-be50a00c15fc admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:48.452143 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31631d3c-f84c-444b-8147-be50a00c15fc admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:48.452279 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:48.457740 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a91304a-160f-4c15-8abb-94a806319f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:48.458390 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91304a-160f-4c15-8abb-94a806319f84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:48.458667 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91304a-160f-4c15-8abb-94a806319f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:48.458941 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a91304a-160f-4c15-8abb-94a806319f84 admin admin] PATH_INFO: `/v3/projects/166dc6362df94359b89547507d2415b2/users/31497b9f3b8f448da0b5edb934bfedfb/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:48.477507 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a91304a-160f-4c15-8abb-94a806319f84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=166dc6362df94359b89547507d2415b2, user_id=31497b9f3b8f448da0b5edb934bfedfb, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:48.478195 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 754/1501] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:48 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:05:48.480315 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a91304a-160f-4c15-8abb-94a806319f84 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:48.489849 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a429690-98db-448b-8ce5-0b0fa21e0d35 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:48.491297 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9a429690-98db-448b-8ce5-0b0fa21e0d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:48.494596 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 748/1502] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:48 2022] PUT /identity/v3/projects/166dc6362df94359b89547507d2415b2/users/31497b9f3b8f448da0b5edb934bfedfb/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:05:48.505737 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a58e7-6565-4f9a-a960-c7645ffe39c5 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:48.506020 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a58e7-6565-4f9a-a960-c7645ffe39c5 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:48.506327 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353a58e7-6565-4f9a-a960-c7645ffe39c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:48.534191 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-353a58e7-6565-4f9a-a960-c7645ffe39c5 None None] MFA Rules not processed for user `31497b9f3b8f448da0b5edb934bfedfb`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:48.535476 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-353a58e7-6565-4f9a-a960-c7645ffe39c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:48.551214 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a429690-98db-448b-8ce5-0b0fa21e0d35 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:48.600117 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 749/1503] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:05:48.600289 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70797c7-65ec-4df7-9335-0f91ddab79ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:48.601032 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70797c7-65ec-4df7-9335-0f91ddab79ad service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:48.601309 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70797c7-65ec-4df7-9335-0f91ddab79ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:48.601587 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70797c7-65ec-4df7-9335-0f91ddab79ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:48.611141 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a58e7-6565-4f9a-a960-c7645ffe39c5 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:48.612510 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-353a58e7-6565-4f9a-a960-c7645ffe39c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:48.619879 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:48.626731 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353a58e7-6565-4f9a-a960-c7645ffe39c5 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:48.630684 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f956edb-3f67-4674-bc44-cb9a439939c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:48.631288 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f956edb-3f67-4674-bc44-cb9a439939c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:48.631574 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f956edb-3f67-4674-bc44-cb9a439939c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:48.631881 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f956edb-3f67-4674-bc44-cb9a439939c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:48.640286 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f956edb-3f67-4674-bc44-cb9a439939c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:48.641704 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f956edb-3f67-4674-bc44-cb9a439939c7 service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:48.651159 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 750/1504] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 20:05:48.666018 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70797c7-65ec-4df7-9335-0f91ddab79ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:48.667514 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70797c7-65ec-4df7-9335-0f91ddab79ad service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:48.676785 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f956edb-3f67-4674-bc44-cb9a439939c7 service cinder] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:48.677861 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 755/1505] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:48 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 20:05:48.752423 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f956edb-3f67-4674-bc44-cb9a439939c7 service cinder] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:48.757153 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04d6b9c-6465-42b8-b2ef-4894cad82f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:48.757800 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d6b9c-6465-42b8-b2ef-4894cad82f97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:48.758079 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d6b9c-6465-42b8-b2ef-4894cad82f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:48.758396 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04d6b9c-6465-42b8-b2ef-4894cad82f97 admin admin] PATH_INFO: `/v3/users/31497b9f3b8f448da0b5edb934bfedfb` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:48.759719 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04d6b9c-6465-42b8-b2ef-4894cad82f97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31497b9f3b8f448da0b5edb934bfedfb)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:48.761650 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04d6b9c-6465-42b8-b2ef-4894cad82f97 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:48.771768 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70797c7-65ec-4df7-9335-0f91ddab79ad service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:48.783261 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70797c7-65ec-4df7-9335-0f91ddab79ad service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:48.788141 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497f5c80-fe07-49b7-bd9d-75540602c57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:48.789354 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497f5c80-fe07-49b7-bd9d-75540602c57f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:48.789705 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497f5c80-fe07-49b7-bd9d-75540602c57f admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:48.790033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497f5c80-fe07-49b7-bd9d-75540602c57f admin admin] PATH_INFO: `/v3/users/4a202356df2f48dc870c18835ea07034` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:48.791356 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497f5c80-fe07-49b7-bd9d-75540602c57f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a202356df2f48dc870c18835ea07034)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:48.793564 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497f5c80-fe07-49b7-bd9d-75540602c57f admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:48.814331 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c04d6b9c-6465-42b8-b2ef-4894cad82f97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31497b9f3b8f448da0b5edb934bfedfb'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:48.830036 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c04d6b9c-6465-42b8-b2ef-4894cad82f97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31497b9f3b8f448da0b5edb934bfedfb'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:48.838433 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c04d6b9c-6465-42b8-b2ef-4894cad82f97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31497b9f3b8f448da0b5edb934bfedfb'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:48.844800 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:48.846768 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c04d6b9c-6465-42b8-b2ef-4894cad82f97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31497b9f3b8f448da0b5edb934bfedfb'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:48.847086 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-497f5c80-fe07-49b7-bd9d-75540602c57f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a202356df2f48dc870c18835ea07034'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:48.847584 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c04d6b9c-6465-42b8-b2ef-4894cad82f97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31497b9f3b8f448da0b5edb934bfedfb'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:48.854534 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 751/1506] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:05:48 2022] DELETE /identity/v3/users/31497b9f3b8f448da0b5edb934bfedfb => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:05:48.859306 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad77e7d-c6b6-4575-a398-fe2df90d3884 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:48.859770 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad77e7d-c6b6-4575-a398-fe2df90d3884 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:48.860214 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad77e7d-c6b6-4575-a398-fe2df90d3884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:48.862582 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-497f5c80-fe07-49b7-bd9d-75540602c57f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a202356df2f48dc870c18835ea07034'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:48.873294 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-497f5c80-fe07-49b7-bd9d-75540602c57f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a202356df2f48dc870c18835ea07034'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:48.873939 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-497f5c80-fe07-49b7-bd9d-75540602c57f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a202356df2f48dc870c18835ea07034'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:48.879717 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-0ad77e7d-c6b6-4575-a398-fe2df90d3884 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:48.880933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-0ad77e7d-c6b6-4575-a398-fe2df90d3884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:48.881234 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-497f5c80-fe07-49b7-bd9d-75540602c57f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a202356df2f48dc870c18835ea07034'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:48.887902 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 756/1507] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:05:48 2022] DELETE /identity/v3/users/4a202356df2f48dc870c18835ea07034 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:05:48.892329 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117c7ee-5713-4f8c-8bf7-48de1981cbfe None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:48.892619 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117c7ee-5713-4f8c-8bf7-48de1981cbfe None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:48.892899 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7117c7ee-5713-4f8c-8bf7-48de1981cbfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:48.913166 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-7117c7ee-5713-4f8c-8bf7-48de1981cbfe None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:48.914489 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7117c7ee-5713-4f8c-8bf7-48de1981cbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:48.946222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 752/1508] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:48.955921 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad77e7d-c6b6-4575-a398-fe2df90d3884 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:48.957309 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-0ad77e7d-c6b6-4575-a398-fe2df90d3884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:48.967505 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:48.979771 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 757/1509] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:05:48.990080 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117c7ee-5713-4f8c-8bf7-48de1981cbfe None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:48.992098 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7117c7ee-5713-4f8c-8bf7-48de1981cbfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:49.021439 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad77e7d-c6b6-4575-a398-fe2df90d3884 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:49.079290 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec4fdc-3c60-4d68-8401-f624a7edc386 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:49.079947 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec4fdc-3c60-4d68-8401-f624a7edc386 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:49.080190 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec4fdc-3c60-4d68-8401-f624a7edc386 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:49.080464 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ec4fdc-3c60-4d68-8401-f624a7edc386 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:49.091383 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7117c7ee-5713-4f8c-8bf7-48de1981cbfe None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:49.098869 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ec4fdc-3c60-4d68-8401-f624a7edc386 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:49.099728 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f76609-17b8-4d7e-9b36-77872b029851 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:49.100386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f76609-17b8-4d7e-9b36-77872b029851 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:49.100697 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f76609-17b8-4d7e-9b36-77872b029851 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:49.100796 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ec4fdc-3c60-4d68-8401-f624a7edc386 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:49.101103 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f76609-17b8-4d7e-9b36-77872b029851 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:49.112184 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f76609-17b8-4d7e-9b36-77872b029851 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:49.113093 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 753/1510] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 20:05:49.113562 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f76609-17b8-4d7e-9b36-77872b029851 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:49.124751 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 758/1511] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 20:05:49.782378 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec4fdc-3c60-4d68-8401-f624a7edc386 service neutron] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:49.784067 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-79ec4fdc-3c60-4d68-8401-f624a7edc386 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:49.801535 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ec4fdc-3c60-4d68-8401-f624a7edc386 service neutron] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:49.807724 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae804f6-0157-451e-a921-d80f3d181d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:49.808548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae804f6-0157-451e-a921-d80f3d181d23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:49.808961 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae804f6-0157-451e-a921-d80f3d181d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:49.809339 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae804f6-0157-451e-a921-d80f3d181d23 admin admin] PATH_INFO: `/v3/projects/166dc6362df94359b89547507d2415b2` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:49.810679 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae804f6-0157-451e-a921-d80f3d181d23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=166dc6362df94359b89547507d2415b2)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:49.812745 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae804f6-0157-451e-a921-d80f3d181d23 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:49.916805 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-8ae804f6-0157-451e-a921-d80f3d181d23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '166dc6362df94359b89547507d2415b2', 'request_id': 'req-8ae804f6-0157-451e-a921-d80f3d181d23'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:49.931637 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-8ae804f6-0157-451e-a921-d80f3d181d23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '166dc6362df94359b89547507d2415b2', 'request_id': 'req-8ae804f6-0157-451e-a921-d80f3d181d23'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:49.935794 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f76609-17b8-4d7e-9b36-77872b029851 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:49.937283 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-b5f76609-17b8-4d7e-9b36-77872b029851 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:49.954609 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-8ae804f6-0157-451e-a921-d80f3d181d23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '166dc6362df94359b89547507d2415b2', 'request_id': 'req-8ae804f6-0157-451e-a921-d80f3d181d23'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:49.976214 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f76609-17b8-4d7e-9b36-77872b029851 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:49.984018 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-8ae804f6-0157-451e-a921-d80f3d181d23 admin admin] The token cache is being invalidate because project 166dc6362df94359b89547507d2415b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:05:49.997185 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-8ae804f6-0157-451e-a921-d80f3d181d23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:49.999321 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-8ae804f6-0157-451e-a921-d80f3d181d23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:50.003088 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 754/1512] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:05:49 2022] DELETE /identity/v3/projects/166dc6362df94359b89547507d2415b2 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:50.008651 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347cbcb-e3bf-4516-a116-a111f0336609 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:50.014540 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347cbcb-e3bf-4516-a116-a111f0336609 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:50.014540 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b347cbcb-e3bf-4516-a116-a111f0336609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:50.043177 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-b347cbcb-e3bf-4516-a116-a111f0336609 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:50.044785 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-b347cbcb-e3bf-4516-a116-a111f0336609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:50.104172 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:50.137601 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 755/1513] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:50.144821 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84d850-f7cd-45b4-b1cb-09bab05d9e19 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:50.145046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84d850-f7cd-45b4-b1cb-09bab05d9e19 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:50.145254 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84d850-f7cd-45b4-b1cb-09bab05d9e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:50.151143 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f8ebb1-17d9-4793-afa9-7399180e1529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:50.154609 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8ebb1-17d9-4793-afa9-7399180e1529 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:50.156728 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8ebb1-17d9-4793-afa9-7399180e1529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:50.157065 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f8ebb1-17d9-4793-afa9-7399180e1529 admin admin] PATH_INFO: `/v3/projects/0b6f1dab6a5c4285a9bfaa9657a9e40a` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:50.161092 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f8ebb1-17d9-4793-afa9-7399180e1529 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b6f1dab6a5c4285a9bfaa9657a9e40a)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:50.169809 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f8ebb1-17d9-4793-afa9-7399180e1529 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:50.189619 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:50.199811 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-ee84d850-f7cd-45b4-b1cb-09bab05d9e19 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:50.202632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-ee84d850-f7cd-45b4-b1cb-09bab05d9e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:50.227410 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 756/1514] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:05:50.232298 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84d850-f7cd-45b4-b1cb-09bab05d9e19 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:50.249131 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee84d850-f7cd-45b4-b1cb-09bab05d9e19 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:50.265445 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e7f8ebb1-17d9-4793-afa9-7399180e1529 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b6f1dab6a5c4285a9bfaa9657a9e40a', 'request_id': 'req-e7f8ebb1-17d9-4793-afa9-7399180e1529'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:50.285104 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e7f8ebb1-17d9-4793-afa9-7399180e1529 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b6f1dab6a5c4285a9bfaa9657a9e40a', 'request_id': 'req-e7f8ebb1-17d9-4793-afa9-7399180e1529'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:50.303233 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e7f8ebb1-17d9-4793-afa9-7399180e1529 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b6f1dab6a5c4285a9bfaa9657a9e40a', 'request_id': 'req-e7f8ebb1-17d9-4793-afa9-7399180e1529'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:50.354332 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e7f8ebb1-17d9-4793-afa9-7399180e1529 admin admin] The token cache is being invalidate because project 0b6f1dab6a5c4285a9bfaa9657a9e40a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:05:50.354631 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e7f8ebb1-17d9-4793-afa9-7399180e1529 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:50.358216 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e7f8ebb1-17d9-4793-afa9-7399180e1529 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:50.361570 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 759/1515] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:05:49 2022] DELETE /identity/v3/projects/0b6f1dab6a5c4285a9bfaa9657a9e40a => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:05:50.365606 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f8ebb1-17d9-4793-afa9-7399180e1529 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:50.367990 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e7f8ebb1-17d9-4793-afa9-7399180e1529 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:50.387321 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7755b6df-64de-4e8b-8c4a-c5201f36a634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:50.388559 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755b6df-64de-4e8b-8c4a-c5201f36a634 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:50.388846 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755b6df-64de-4e8b-8c4a-c5201f36a634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:50.389121 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7755b6df-64de-4e8b-8c4a-c5201f36a634 admin admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:50.389801 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7755b6df-64de-4e8b-8c4a-c5201f36a634 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:05:50.391116 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7755b6df-64de-4e8b-8c4a-c5201f36a634 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:50.393995 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7755b6df-64de-4e8b-8c4a-c5201f36a634 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:50.422394 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 757/1516] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:05:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 20:05:50.425784 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7755b6df-64de-4e8b-8c4a-c5201f36a634 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:50.430145 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-7755b6df-64de-4e8b-8c4a-c5201f36a634 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:50.469397 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7755b6df-64de-4e8b-8c4a-c5201f36a634 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:50.476240 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc91fd-6f75-4ef2-b9a5-34faec3f1bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:50.476918 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc91fd-6f75-4ef2-b9a5-34faec3f1bae admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:50.477303 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc91fd-6f75-4ef2-b9a5-34faec3f1bae admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:50.477770 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09fc91fd-6f75-4ef2-b9a5-34faec3f1bae admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:50.478858 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc91fd-6f75-4ef2-b9a5-34faec3f1bae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:50.482465 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09fc91fd-6f75-4ef2-b9a5-34faec3f1bae admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:50.521894 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f8ebb1-17d9-4793-afa9-7399180e1529 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:50.532891 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0141379-8b0b-47a7-8064-b43f7cb4386b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:50.532891 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0141379-8b0b-47a7-8064-b43f7cb4386b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:50.532891 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0141379-8b0b-47a7-8064-b43f7cb4386b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:50.537119 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0141379-8b0b-47a7-8064-b43f7cb4386b admin admin] PATH_INFO: `/v3/users/ce78583d1272418fb5adc985438a5ba5` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:50.540941 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 758/1517] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:05:50 2022] POST /identity/v3/projects => generated 389 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:50.543995 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0141379-8b0b-47a7-8064-b43f7cb4386b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce78583d1272418fb5adc985438a5ba5)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:50.545371 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc91fd-6f75-4ef2-b9a5-34faec3f1bae admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:50.545983 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0141379-8b0b-47a7-8064-b43f7cb4386b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:50.666198 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e0141379-8b0b-47a7-8064-b43f7cb4386b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce78583d1272418fb5adc985438a5ba5'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:50.695629 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e0141379-8b0b-47a7-8064-b43f7cb4386b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce78583d1272418fb5adc985438a5ba5'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:50.705844 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e0141379-8b0b-47a7-8064-b43f7cb4386b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce78583d1272418fb5adc985438a5ba5'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:50.706766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e0141379-8b0b-47a7-8064-b43f7cb4386b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce78583d1272418fb5adc985438a5ba5'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:50.716816 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09fc91fd-6f75-4ef2-b9a5-34faec3f1bae admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:50.718773 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e0141379-8b0b-47a7-8064-b43f7cb4386b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce78583d1272418fb5adc985438a5ba5'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:50.732607 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366455ab-6d37-43bb-93b1-16ff5026f9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:50.736535 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 760/1518] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:05:50 2022] DELETE /identity/v3/users/ce78583d1272418fb5adc985438a5ba5 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:50.736702 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366455ab-6d37-43bb-93b1-16ff5026f9e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:50.736702 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366455ab-6d37-43bb-93b1-16ff5026f9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:50.736702 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366455ab-6d37-43bb-93b1-16ff5026f9e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:50.736702 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366455ab-6d37-43bb-93b1-16ff5026f9e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:50.739501 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366455ab-6d37-43bb-93b1-16ff5026f9e7 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:50.774107 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 759/1519] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:05:50 2022] POST /identity/v3/users => generated 295 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:05:50.776118 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0141379-8b0b-47a7-8064-b43f7cb4386b admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:50.778423 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e0141379-8b0b-47a7-8064-b43f7cb4386b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:50.822803 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:50.910416 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0141379-8b0b-47a7-8064-b43f7cb4386b admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:50.918372 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb692a6-d291-42c2-9f80-560f4f716fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:50.918982 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb692a6-d291-42c2-9f80-560f4f716fa6 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:50.919397 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb692a6-d291-42c2-9f80-560f4f716fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:50.919699 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb692a6-d291-42c2-9f80-560f4f716fa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:50.921216 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb692a6-d291-42c2-9f80-560f4f716fa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:50.922970 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb692a6-d291-42c2-9f80-560f4f716fa6 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:50.936015 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 761/1520] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 20:05:50.940114 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366455ab-6d37-43bb-93b1-16ff5026f9e7 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:50.957117 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366455ab-6d37-43bb-93b1-16ff5026f9e7 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:50.963071 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e279a8-fe26-40a9-8ef4-fa97cb6b6100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:50.963829 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e279a8-fe26-40a9-8ef4-fa97cb6b6100 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:50.964604 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e279a8-fe26-40a9-8ef4-fa97cb6b6100 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:50.964604 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e279a8-fe26-40a9-8ef4-fa97cb6b6100 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:50.965198 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e279a8-fe26-40a9-8ef4-fa97cb6b6100 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:50.966962 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e279a8-fe26-40a9-8ef4-fa97cb6b6100 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:50.979406 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 760/1521] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:50.983533 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb692a6-d291-42c2-9f80-560f4f716fa6 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:50.994478 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb692a6-d291-42c2-9f80-560f4f716fa6 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:50.999754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dab1a3-a12e-44e0-ae2c-aea930d0fdc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:51.000552 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dab1a3-a12e-44e0-ae2c-aea930d0fdc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:51.000910 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dab1a3-a12e-44e0-ae2c-aea930d0fdc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:51.001258 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dab1a3-a12e-44e0-ae2c-aea930d0fdc3 admin admin] PATH_INFO: `/v3/projects/05112a5d8e634e34a2eed263b6b3db76/users/9c6a3f6397c44eeb823df053954900e8/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:51.026629 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dab1a3-a12e-44e0-ae2c-aea930d0fdc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05112a5d8e634e34a2eed263b6b3db76, user_id=9c6a3f6397c44eeb823df053954900e8, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:51.029586 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dab1a3-a12e-44e0-ae2c-aea930d0fdc3 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:51.045786 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 762/1522] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:50 2022] PUT /identity/v3/projects/05112a5d8e634e34a2eed263b6b3db76/users/9c6a3f6397c44eeb823df053954900e8/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:05:51.056382 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7627954-5470-42e4-be76-8a46566f6bc5 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:51.056640 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7627954-5470-42e4-be76-8a46566f6bc5 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:51.056875 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7627954-5470-42e4-be76-8a46566f6bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:51.136090 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-c7627954-5470-42e4-be76-8a46566f6bc5 None None] MFA Rules not processed for user `9c6a3f6397c44eeb823df053954900e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:51.138314 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-c7627954-5470-42e4-be76-8a46566f6bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:51.210468 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:51.271051 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 761/1523] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:51 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:51.285172 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dab1a3-a12e-44e0-ae2c-aea930d0fdc3 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:51.289452 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-96dab1a3-a12e-44e0-ae2c-aea930d0fdc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:51.403746 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dab1a3-a12e-44e0-ae2c-aea930d0fdc3 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:51.408199 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c570ed9-ed44-4ad5-af24-81fa427c2527 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:51.409014 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c570ed9-ed44-4ad5-af24-81fa427c2527 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:51.409306 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c570ed9-ed44-4ad5-af24-81fa427c2527 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:51.409579 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c570ed9-ed44-4ad5-af24-81fa427c2527 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:51.424569 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c570ed9-ed44-4ad5-af24-81fa427c2527 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:51.426699 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c570ed9-ed44-4ad5-af24-81fa427c2527 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:51.448340 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 763/1524] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:51 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:05:52.555984 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7627954-5470-42e4-be76-8a46566f6bc5 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:52.558963 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-c7627954-5470-42e4-be76-8a46566f6bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:52.700363 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7627954-5470-42e4-be76-8a46566f6bc5 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:52.707541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed95d146-69b2-40a1-87cf-14bc9ac98be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:52.708413 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95d146-69b2-40a1-87cf-14bc9ac98be6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:52.708891 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95d146-69b2-40a1-87cf-14bc9ac98be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:52.709225 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed95d146-69b2-40a1-87cf-14bc9ac98be6 admin admin] PATH_INFO: `/v3/projects/a084917827a64336b7537e0e4cc8c3ee` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:52.710577 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed95d146-69b2-40a1-87cf-14bc9ac98be6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a084917827a64336b7537e0e4cc8c3ee)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:52.713233 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed95d146-69b2-40a1-87cf-14bc9ac98be6 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:52.783678 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ed95d146-69b2-40a1-87cf-14bc9ac98be6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a084917827a64336b7537e0e4cc8c3ee', 'request_id': 'req-ed95d146-69b2-40a1-87cf-14bc9ac98be6'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:52.795775 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ed95d146-69b2-40a1-87cf-14bc9ac98be6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a084917827a64336b7537e0e4cc8c3ee', 'request_id': 'req-ed95d146-69b2-40a1-87cf-14bc9ac98be6'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:52.806134 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ed95d146-69b2-40a1-87cf-14bc9ac98be6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a084917827a64336b7537e0e4cc8c3ee', 'request_id': 'req-ed95d146-69b2-40a1-87cf-14bc9ac98be6'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:52.824427 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ed95d146-69b2-40a1-87cf-14bc9ac98be6 admin admin] The token cache is being invalidate because project a084917827a64336b7537e0e4cc8c3ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:05:52.824733 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ed95d146-69b2-40a1-87cf-14bc9ac98be6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:52.825428 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ed95d146-69b2-40a1-87cf-14bc9ac98be6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:52.831002 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 762/1525] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:05:52 2022] DELETE /identity/v3/projects/a084917827a64336b7537e0e4cc8c3ee => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:52.833201 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff3fa4-e805-4304-83f9-bbccb7be6a57 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:52.833668 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff3fa4-e805-4304-83f9-bbccb7be6a57 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:52.833760 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeff3fa4-e805-4304-83f9-bbccb7be6a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:52.871290 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-eeff3fa4-e805-4304-83f9-bbccb7be6a57 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:52.872774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-eeff3fa4-e805-4304-83f9-bbccb7be6a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:52.968373 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 764/1526] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:52.969997 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657b108-ebe7-4b27-a27c-379d594589c3 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:52.970317 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657b108-ebe7-4b27-a27c-379d594589c3 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:52.970602 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657b108-ebe7-4b27-a27c-379d594589c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:52.994679 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-2657b108-ebe7-4b27-a27c-379d594589c3 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:52.996076 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-2657b108-ebe7-4b27-a27c-379d594589c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:53.008501 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 763/1527] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:53.025168 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeff3fa4-e805-4304-83f9-bbccb7be6a57 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:53.035645 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeff3fa4-e805-4304-83f9-bbccb7be6a57 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:53.040947 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec3dd75-ea0f-44ab-894c-3dd3d84afbe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:53.041568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec3dd75-ea0f-44ab-894c-3dd3d84afbe0 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:53.041842 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec3dd75-ea0f-44ab-894c-3dd3d84afbe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:53.042114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec3dd75-ea0f-44ab-894c-3dd3d84afbe0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:53.042813 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec3dd75-ea0f-44ab-894c-3dd3d84afbe0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:05:53.043252 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec3dd75-ea0f-44ab-894c-3dd3d84afbe0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:53.044907 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec3dd75-ea0f-44ab-894c-3dd3d84afbe0 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:53.061443 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 765/1528] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:05:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:05:53.062645 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657b108-ebe7-4b27-a27c-379d594589c3 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:53.072524 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657b108-ebe7-4b27-a27c-379d594589c3 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:53.077520 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe4c5b4-75cf-4a6e-b8e4-0443066d260d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:53.078263 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4c5b4-75cf-4a6e-b8e4-0443066d260d admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:53.078645 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4c5b4-75cf-4a6e-b8e4-0443066d260d admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:53.078984 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe4c5b4-75cf-4a6e-b8e4-0443066d260d admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:53.079819 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe4c5b4-75cf-4a6e-b8e4-0443066d260d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:53.081834 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe4c5b4-75cf-4a6e-b8e4-0443066d260d admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:53.111283 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 764/1529] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:05:53 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:05:53.115799 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec3dd75-ea0f-44ab-894c-3dd3d84afbe0 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:53.126386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec3dd75-ea0f-44ab-894c-3dd3d84afbe0 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:53.138954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:53.221028 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9adc4d5-23e3-475d-99e4-c17265ec9af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:53.222043 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9adc4d5-23e3-475d-99e4-c17265ec9af3 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:53.222496 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9adc4d5-23e3-475d-99e4-c17265ec9af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:53.223325 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9adc4d5-23e3-475d-99e4-c17265ec9af3 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:53.224166 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9adc4d5-23e3-475d-99e4-c17265ec9af3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:53.226471 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9adc4d5-23e3-475d-99e4-c17265ec9af3 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:53.263589 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 766/1530] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:05:53 2022] POST /identity/v3/users => generated 291 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:53.267576 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe4c5b4-75cf-4a6e-b8e4-0443066d260d admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:53.286459 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe4c5b4-75cf-4a6e-b8e4-0443066d260d admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:53.291324 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3651bfac-2764-4f9e-97e1-0da5376e2a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:53.292018 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651bfac-2764-4f9e-97e1-0da5376e2a0b admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:53.292307 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651bfac-2764-4f9e-97e1-0da5376e2a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:53.292580 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3651bfac-2764-4f9e-97e1-0da5376e2a0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:53.293238 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3651bfac-2764-4f9e-97e1-0da5376e2a0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:53.294870 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3651bfac-2764-4f9e-97e1-0da5376e2a0b admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:53.311652 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 765/1531] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:53.314988 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9adc4d5-23e3-475d-99e4-c17265ec9af3 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:53.325327 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9adc4d5-23e3-475d-99e4-c17265ec9af3 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:53.330071 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9708571-d8ec-4fe6-8160-15d67e376fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:53.330723 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9708571-d8ec-4fe6-8160-15d67e376fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:53.331010 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9708571-d8ec-4fe6-8160-15d67e376fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:53.331290 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9708571-d8ec-4fe6-8160-15d67e376fa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:53.331950 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9708571-d8ec-4fe6-8160-15d67e376fa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:53.333551 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9708571-d8ec-4fe6-8160-15d67e376fa0 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:53.346165 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 767/1532] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:53.350529 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3651bfac-2764-4f9e-97e1-0da5376e2a0b admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:53.361187 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3651bfac-2764-4f9e-97e1-0da5376e2a0b admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:53.368744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54587cd-5208-4e40-8e4c-9e34c073b62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:53.369599 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54587cd-5208-4e40-8e4c-9e34c073b62c admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:53.369941 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54587cd-5208-4e40-8e4c-9e34c073b62c admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:53.370227 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54587cd-5208-4e40-8e4c-9e34c073b62c admin admin] PATH_INFO: `/v3/projects/9cf663d84e1d49969db58d44924af63e/users/43a86d52117d4fc3a4acc22b24c3986b/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:53.384887 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:53.389833 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54587cd-5208-4e40-8e4c-9e34c073b62c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cf663d84e1d49969db58d44924af63e, user_id=43a86d52117d4fc3a4acc22b24c3986b, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:53.393712 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54587cd-5208-4e40-8e4c-9e34c073b62c admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:53.419636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 766/1533] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:53 2022] PUT /identity/v3/projects/9cf663d84e1d49969db58d44924af63e/users/43a86d52117d4fc3a4acc22b24c3986b/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:05:53.421327 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ec416-f4b2-4697-8ac4-79de0ea7fdbc None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:53.421629 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ec416-f4b2-4697-8ac4-79de0ea7fdbc None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:53.422003 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ec416-f4b2-4697-8ac4-79de0ea7fdbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:53.575775 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-289ec416-f4b2-4697-8ac4-79de0ea7fdbc None None] MFA Rules not processed for user `43a86d52117d4fc3a4acc22b24c3986b`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:53.576988 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-289ec416-f4b2-4697-8ac4-79de0ea7fdbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:53.643103 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 768/1534] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:53.648901 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54587cd-5208-4e40-8e4c-9e34c073b62c admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:53.717282 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54587cd-5208-4e40-8e4c-9e34c073b62c admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:53.721928 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a924964b-1871-4356-aa0e-3eb56ec6c29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:53.722567 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924964b-1871-4356-aa0e-3eb56ec6c29d admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:53.722847 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924964b-1871-4356-aa0e-3eb56ec6c29d admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:53.723115 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a924964b-1871-4356-aa0e-3eb56ec6c29d admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:53.723764 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a924964b-1871-4356-aa0e-3eb56ec6c29d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:53.726269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a924964b-1871-4356-aa0e-3eb56ec6c29d admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:53.755719 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 767/1535] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:05:53 2022] POST /identity/v3/projects => generated 377 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:53.760128 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ec416-f4b2-4697-8ac4-79de0ea7fdbc None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:53.771200 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ec416-f4b2-4697-8ac4-79de0ea7fdbc None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:53.848297 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00a80d-41ad-43b9-8587-ad6c738f1611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:53.849204 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00a80d-41ad-43b9-8587-ad6c738f1611 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:53.849683 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00a80d-41ad-43b9-8587-ad6c738f1611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:53.850046 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00a80d-41ad-43b9-8587-ad6c738f1611 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:53.850850 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab00a80d-41ad-43b9-8587-ad6c738f1611 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:53.852871 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab00a80d-41ad-43b9-8587-ad6c738f1611 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:53.877680 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 769/1536] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:05:53 2022] POST /identity/v3/users => generated 295 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:53.879226 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a924964b-1871-4356-aa0e-3eb56ec6c29d admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:53.889804 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a924964b-1871-4356-aa0e-3eb56ec6c29d admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:53.894425 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a40a5-5921-413b-bf2b-2663af5f2a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:53.895057 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a40a5-5921-413b-bf2b-2663af5f2a78 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:53.895335 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a40a5-5921-413b-bf2b-2663af5f2a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:53.895603 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a40a5-5921-413b-bf2b-2663af5f2a78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:53.896278 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146a40a5-5921-413b-bf2b-2663af5f2a78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:53.898540 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146a40a5-5921-413b-bf2b-2663af5f2a78 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:53.909161 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 768/1537] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:53.914139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00a80d-41ad-43b9-8587-ad6c738f1611 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:53.924343 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00a80d-41ad-43b9-8587-ad6c738f1611 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:53.929306 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6d3f7-2f0a-4850-9558-1f89510d0a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:53.929983 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6d3f7-2f0a-4850-9558-1f89510d0a74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:53.930293 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6d3f7-2f0a-4850-9558-1f89510d0a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:53.930561 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d6d3f7-2f0a-4850-9558-1f89510d0a74 admin admin] PATH_INFO: `/v3/projects/fde08925e37e4cfb9aeab915b74bf089/users/6419908f34f241c38c97f5064218c908/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:53.956043 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:53.960527 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d6d3f7-2f0a-4850-9558-1f89510d0a74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fde08925e37e4cfb9aeab915b74bf089, user_id=6419908f34f241c38c97f5064218c908, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:53.963883 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d6d3f7-2f0a-4850-9558-1f89510d0a74 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:53.979249 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 770/1538] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:53 2022] PUT /identity/v3/projects/fde08925e37e4cfb9aeab915b74bf089/users/6419908f34f241c38c97f5064218c908/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:53.983199 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a40a5-5921-413b-bf2b-2663af5f2a78 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:53.993222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a40a5-5921-413b-bf2b-2663af5f2a78 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:54.050906 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a2f8ee-9add-440b-88c3-d112433df5a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:54.051526 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2f8ee-9add-440b-88c3-d112433df5a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:54.051811 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2f8ee-9add-440b-88c3-d112433df5a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:54.052086 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2f8ee-9add-440b-88c3-d112433df5a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:54.052736 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a2f8ee-9add-440b-88c3-d112433df5a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:54.054362 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a2f8ee-9add-440b-88c3-d112433df5a7 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:54.064247 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 769/1539] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:53 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:54.068586 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6d3f7-2f0a-4850-9558-1f89510d0a74 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:54.078984 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d6d3f7-2f0a-4850-9558-1f89510d0a74 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:54.083473 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aaa57b-d913-4cc3-a81a-190393b99449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:54.084117 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aaa57b-d913-4cc3-a81a-190393b99449 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:54.084394 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aaa57b-d913-4cc3-a81a-190393b99449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:54.084663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aaa57b-d913-4cc3-a81a-190393b99449 admin admin] PATH_INFO: `/v3/projects/fde08925e37e4cfb9aeab915b74bf089/users/6419908f34f241c38c97f5064218c908/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:54.087088 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aaa57b-d913-4cc3-a81a-190393b99449 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fde08925e37e4cfb9aeab915b74bf089, user_id=6419908f34f241c38c97f5064218c908, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:54.089688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aaa57b-d913-4cc3-a81a-190393b99449 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:54.099592 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 771/1540] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:54 2022] PUT /identity/v3/projects/fde08925e37e4cfb9aeab915b74bf089/users/6419908f34f241c38c97f5064218c908/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:05:54.109924 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887fafd0-f90e-4138-9809-ce9888ba1a68 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:54.110330 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887fafd0-f90e-4138-9809-ce9888ba1a68 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:54.110673 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887fafd0-f90e-4138-9809-ce9888ba1a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:54.137134 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-887fafd0-f90e-4138-9809-ce9888ba1a68 None None] MFA Rules not processed for user `6419908f34f241c38c97f5064218c908`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:54.138353 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-887fafd0-f90e-4138-9809-ce9888ba1a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:54.185555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:54.207574 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 770/1541] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:54 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:54.218997 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aaa57b-d913-4cc3-a81a-190393b99449 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:54.220670 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-75aaa57b-d913-4cc3-a81a-190393b99449 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:54.278817 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aaa57b-d913-4cc3-a81a-190393b99449 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:54.282477 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d760a-4e89-4615-a4a6-f2b72c13cefb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:54.283080 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d760a-4e89-4615-a4a6-f2b72c13cefb service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:54.283362 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d760a-4e89-4615-a4a6-f2b72c13cefb service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:54.283639 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d760a-4e89-4615-a4a6-f2b72c13cefb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:54.293325 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d760a-4e89-4615-a4a6-f2b72c13cefb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:54.294802 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d760a-4e89-4615-a4a6-f2b72c13cefb service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:54.309170 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 772/1542] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:54 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:05:54.682014 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887fafd0-f90e-4138-9809-ce9888ba1a68 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:54.747122 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887fafd0-f90e-4138-9809-ce9888ba1a68 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:54.751644 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046901c3-4095-41c8-8649-0ccdd5c7b36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:54.752303 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046901c3-4095-41c8-8649-0ccdd5c7b36b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:54.752582 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046901c3-4095-41c8-8649-0ccdd5c7b36b admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:54.752856 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046901c3-4095-41c8-8649-0ccdd5c7b36b admin admin] PATH_INFO: `/v3/users/43a86d52117d4fc3a4acc22b24c3986b` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:54.753971 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046901c3-4095-41c8-8649-0ccdd5c7b36b admin admin] RBAC: Authorizing `identity:delete_user(user_id=43a86d52117d4fc3a4acc22b24c3986b)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:54.755964 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046901c3-4095-41c8-8649-0ccdd5c7b36b admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:54.800955 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-046901c3-4095-41c8-8649-0ccdd5c7b36b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43a86d52117d4fc3a4acc22b24c3986b'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:54.813399 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-046901c3-4095-41c8-8649-0ccdd5c7b36b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43a86d52117d4fc3a4acc22b24c3986b'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:54.821368 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-046901c3-4095-41c8-8649-0ccdd5c7b36b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43a86d52117d4fc3a4acc22b24c3986b'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:54.829055 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-046901c3-4095-41c8-8649-0ccdd5c7b36b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43a86d52117d4fc3a4acc22b24c3986b'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:54.829778 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-046901c3-4095-41c8-8649-0ccdd5c7b36b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43a86d52117d4fc3a4acc22b24c3986b'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:54.836694 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 771/1543] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:05:54 2022] DELETE /identity/v3/users/43a86d52117d4fc3a4acc22b24c3986b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:54.841297 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5157c6a3-ab74-43e7-8ea2-b29148d2d059 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:54.841607 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5157c6a3-ab74-43e7-8ea2-b29148d2d059 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:54.841908 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5157c6a3-ab74-43e7-8ea2-b29148d2d059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:54.860548 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-5157c6a3-ab74-43e7-8ea2-b29148d2d059 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:54.861734 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5157c6a3-ab74-43e7-8ea2-b29148d2d059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:54.922700 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 773/1544] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:54.933039 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046901c3-4095-41c8-8649-0ccdd5c7b36b admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:54.934565 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-046901c3-4095-41c8-8649-0ccdd5c7b36b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:54.996149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046901c3-4095-41c8-8649-0ccdd5c7b36b admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:54.999736 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023b26e-8114-4a3d-9c28-cc32ac28aea6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:55.000346 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023b26e-8114-4a3d-9c28-cc32ac28aea6 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:55.000626 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023b26e-8114-4a3d-9c28-cc32ac28aea6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:55.000900 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d023b26e-8114-4a3d-9c28-cc32ac28aea6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:55.015437 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023b26e-8114-4a3d-9c28-cc32ac28aea6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:55.017653 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d023b26e-8114-4a3d-9c28-cc32ac28aea6 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:55.030271 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 772/1545] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:05:55.314124 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5157c6a3-ab74-43e7-8ea2-b29148d2d059 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:55.315656 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5157c6a3-ab74-43e7-8ea2-b29148d2d059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:55.334036 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5157c6a3-ab74-43e7-8ea2-b29148d2d059 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:55.340516 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d80cb2-c0bc-4c76-ab9a-6d5c69177a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:55.341350 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d80cb2-c0bc-4c76-ab9a-6d5c69177a31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:55.341730 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d80cb2-c0bc-4c76-ab9a-6d5c69177a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:55.341893 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d80cb2-c0bc-4c76-ab9a-6d5c69177a31 admin admin] PATH_INFO: `/v3/projects/9cf663d84e1d49969db58d44924af63e` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:55.343381 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d80cb2-c0bc-4c76-ab9a-6d5c69177a31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cf663d84e1d49969db58d44924af63e)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:55.353773 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d80cb2-c0bc-4c76-ab9a-6d5c69177a31 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:55.400315 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e4d80cb2-c0bc-4c76-ab9a-6d5c69177a31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cf663d84e1d49969db58d44924af63e', 'request_id': 'req-e4d80cb2-c0bc-4c76-ab9a-6d5c69177a31'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:55.407584 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e4d80cb2-c0bc-4c76-ab9a-6d5c69177a31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cf663d84e1d49969db58d44924af63e', 'request_id': 'req-e4d80cb2-c0bc-4c76-ab9a-6d5c69177a31'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:55.414083 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e4d80cb2-c0bc-4c76-ab9a-6d5c69177a31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cf663d84e1d49969db58d44924af63e', 'request_id': 'req-e4d80cb2-c0bc-4c76-ab9a-6d5c69177a31'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:55.425073 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e4d80cb2-c0bc-4c76-ab9a-6d5c69177a31 admin admin] The token cache is being invalidate because project 9cf663d84e1d49969db58d44924af63e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:05:55.425355 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e4d80cb2-c0bc-4c76-ab9a-6d5c69177a31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:55.426007 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e4d80cb2-c0bc-4c76-ab9a-6d5c69177a31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:55.427501 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 774/1546] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:05:55 2022] DELETE /identity/v3/projects/9cf663d84e1d49969db58d44924af63e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:55.431611 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023b26e-8114-4a3d-9c28-cc32ac28aea6 service neutron] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:55.441301 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d023b26e-8114-4a3d-9c28-cc32ac28aea6 service neutron] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:55.446268 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:55.499182 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3c0b8a-a200-4d0f-967e-b52ed4a910b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:55.499829 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c0b8a-a200-4d0f-967e-b52ed4a910b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:55.500102 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c0b8a-a200-4d0f-967e-b52ed4a910b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:55.500378 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3c0b8a-a200-4d0f-967e-b52ed4a910b0 admin admin] PATH_INFO: `/v3/users/6419908f34f241c38c97f5064218c908` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:55.501475 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3c0b8a-a200-4d0f-967e-b52ed4a910b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6419908f34f241c38c97f5064218c908)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:55.503403 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3c0b8a-a200-4d0f-967e-b52ed4a910b0 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:55.550733 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-0e3c0b8a-a200-4d0f-967e-b52ed4a910b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6419908f34f241c38c97f5064218c908'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:55.562915 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-0e3c0b8a-a200-4d0f-967e-b52ed4a910b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6419908f34f241c38c97f5064218c908'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:55.570535 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-0e3c0b8a-a200-4d0f-967e-b52ed4a910b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6419908f34f241c38c97f5064218c908'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:55.577933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-0e3c0b8a-a200-4d0f-967e-b52ed4a910b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6419908f34f241c38c97f5064218c908'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:55.578635 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-0e3c0b8a-a200-4d0f-967e-b52ed4a910b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6419908f34f241c38c97f5064218c908'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:55.584657 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 773/1547] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:05:55 2022] DELETE /identity/v3/users/6419908f34f241c38c97f5064218c908 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:55.910705 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d80cb2-c0bc-4c76-ab9a-6d5c69177a31 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:55.912321 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e4d80cb2-c0bc-4c76-ab9a-6d5c69177a31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:55.922198 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:55.926300 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d80cb2-c0bc-4c76-ab9a-6d5c69177a31 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:55.984411 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65223fcd-369f-4b6c-9a95-45cf1e597faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:55.985089 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65223fcd-369f-4b6c-9a95-45cf1e597faf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:55.985373 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65223fcd-369f-4b6c-9a95-45cf1e597faf admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:55.985658 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65223fcd-369f-4b6c-9a95-45cf1e597faf admin admin] PATH_INFO: `/v3/projects/fde08925e37e4cfb9aeab915b74bf089` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:55.986800 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65223fcd-369f-4b6c-9a95-45cf1e597faf admin admin] RBAC: Authorizing `identity:delete_project(project_id=fde08925e37e4cfb9aeab915b74bf089)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:55.988699 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65223fcd-369f-4b6c-9a95-45cf1e597faf admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:56.038539 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-65223fcd-369f-4b6c-9a95-45cf1e597faf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fde08925e37e4cfb9aeab915b74bf089', 'request_id': 'req-65223fcd-369f-4b6c-9a95-45cf1e597faf'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:56.046886 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-65223fcd-369f-4b6c-9a95-45cf1e597faf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fde08925e37e4cfb9aeab915b74bf089', 'request_id': 'req-65223fcd-369f-4b6c-9a95-45cf1e597faf'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:56.052780 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-65223fcd-369f-4b6c-9a95-45cf1e597faf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fde08925e37e4cfb9aeab915b74bf089', 'request_id': 'req-65223fcd-369f-4b6c-9a95-45cf1e597faf'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:56.065666 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-65223fcd-369f-4b6c-9a95-45cf1e597faf admin admin] The token cache is being invalidate because project fde08925e37e4cfb9aeab915b74bf089 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:05:56.065958 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-65223fcd-369f-4b6c-9a95-45cf1e597faf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:56.066658 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-65223fcd-369f-4b6c-9a95-45cf1e597faf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:05:56.068308 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 775/1548] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:05:55 2022] DELETE /identity/v3/projects/fde08925e37e4cfb9aeab915b74bf089 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:56.074789 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830ad5a-3487-485b-9c9d-d4acae5c45f9 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:56.075256 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830ad5a-3487-485b-9c9d-d4acae5c45f9 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:56.075458 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4830ad5a-3487-485b-9c9d-d4acae5c45f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:56.095164 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-4830ad5a-3487-485b-9c9d-d4acae5c45f9 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:56.096365 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-4830ad5a-3487-485b-9c9d-d4acae5c45f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:56.158706 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 774/1549] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:56.169040 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc190f4-4bc2-439a-bb4c-6358226bdcea None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:56.169330 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc190f4-4bc2-439a-bb4c-6358226bdcea None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:56.169605 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc190f4-4bc2-439a-bb4c-6358226bdcea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:56.189261 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-8fc190f4-4bc2-439a-bb4c-6358226bdcea None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:56.190754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-8fc190f4-4bc2-439a-bb4c-6358226bdcea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:56.199826 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 776/1550] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:56.203981 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830ad5a-3487-485b-9c9d-d4acae5c45f9 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:56.214275 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4830ad5a-3487-485b-9c9d-d4acae5c45f9 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:56.219025 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708b9c0-ff15-4723-99a9-a15915978ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:56.219635 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708b9c0-ff15-4723-99a9-a15915978ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:56.219918 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708b9c0-ff15-4723-99a9-a15915978ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:56.220195 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e708b9c0-ff15-4723-99a9-a15915978ae3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:56.220854 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e708b9c0-ff15-4723-99a9-a15915978ae3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:05:56.221162 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e708b9c0-ff15-4723-99a9-a15915978ae3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:56.222792 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e708b9c0-ff15-4723-99a9-a15915978ae3 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:56.235512 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 775/1551] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:05:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:05:56.239206 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc190f4-4bc2-439a-bb4c-6358226bdcea None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:56.252548 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc190f4-4bc2-439a-bb4c-6358226bdcea None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:56.257828 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee599a21-b708-4e58-b9f7-b5bb727de50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:56.258478 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee599a21-b708-4e58-b9f7-b5bb727de50e admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:56.258757 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee599a21-b708-4e58-b9f7-b5bb727de50e admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:56.259051 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee599a21-b708-4e58-b9f7-b5bb727de50e admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:56.260609 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee599a21-b708-4e58-b9f7-b5bb727de50e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:56.261619 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee599a21-b708-4e58-b9f7-b5bb727de50e admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:56.296059 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 777/1552] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:05:56 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:05:56.297182 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708b9c0-ff15-4723-99a9-a15915978ae3 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:56.307764 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e708b9c0-ff15-4723-99a9-a15915978ae3 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:56.319812 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:56.377701 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859a420-713a-4e89-ba3c-ac13f759262c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:56.378354 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859a420-713a-4e89-ba3c-ac13f759262c admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:56.378663 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859a420-713a-4e89-ba3c-ac13f759262c admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:56.379014 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f859a420-713a-4e89-ba3c-ac13f759262c admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:56.379777 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f859a420-713a-4e89-ba3c-ac13f759262c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:56.382534 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f859a420-713a-4e89-ba3c-ac13f759262c admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:56.409480 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 776/1553] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:05:56 2022] POST /identity/v3/users => generated 288 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:56.413410 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee599a21-b708-4e58-b9f7-b5bb727de50e admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:56.423670 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee599a21-b708-4e58-b9f7-b5bb727de50e admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:56.428174 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a752db-1423-44d4-9fdd-c0528a11ab25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:56.429453 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a752db-1423-44d4-9fdd-c0528a11ab25 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:56.429730 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a752db-1423-44d4-9fdd-c0528a11ab25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:56.430003 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a752db-1423-44d4-9fdd-c0528a11ab25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:56.430680 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a752db-1423-44d4-9fdd-c0528a11ab25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:56.432377 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a752db-1423-44d4-9fdd-c0528a11ab25 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:56.444613 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 778/1554] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:56.448564 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859a420-713a-4e89-ba3c-ac13f759262c admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:56.458514 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f859a420-713a-4e89-ba3c-ac13f759262c admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:56.463622 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0e1732-3a26-4b7a-84d6-ecc158b72c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:56.464217 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e1732-3a26-4b7a-84d6-ecc158b72c86 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:56.464515 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e1732-3a26-4b7a-84d6-ecc158b72c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:56.464771 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0e1732-3a26-4b7a-84d6-ecc158b72c86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:56.465417 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0e1732-3a26-4b7a-84d6-ecc158b72c86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:56.467037 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0e1732-3a26-4b7a-84d6-ecc158b72c86 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:56.478153 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 777/1555] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:56.481985 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a752db-1423-44d4-9fdd-c0528a11ab25 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:56.492392 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a752db-1423-44d4-9fdd-c0528a11ab25 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:56.497089 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b3c62-3999-4834-9123-3124c6881f66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:56.497742 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b3c62-3999-4834-9123-3124c6881f66 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:56.498046 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b3c62-3999-4834-9123-3124c6881f66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:56.498386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6b3c62-3999-4834-9123-3124c6881f66 admin admin] PATH_INFO: `/v3/projects/6ac0b628da0b4de1bac868f0cfd39236/users/f8fc5e88de3549fb860fa99f5adb0cb0/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:56.513213 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6b3c62-3999-4834-9123-3124c6881f66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ac0b628da0b4de1bac868f0cfd39236, user_id=f8fc5e88de3549fb860fa99f5adb0cb0, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:56.515913 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6b3c62-3999-4834-9123-3124c6881f66 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:56.529453 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 779/1556] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:56 2022] PUT /identity/v3/projects/6ac0b628da0b4de1bac868f0cfd39236/users/f8fc5e88de3549fb860fa99f5adb0cb0/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:05:56.540055 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73a397-cde3-498b-be16-31966737c0a4 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:56.540340 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73a397-cde3-498b-be16-31966737c0a4 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:56.540638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73a397-cde3-498b-be16-31966737c0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:56.568083 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-2d73a397-cde3-498b-be16-31966737c0a4 None None] MFA Rules not processed for user `f8fc5e88de3549fb860fa99f5adb0cb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:56.570131 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-2d73a397-cde3-498b-be16-31966737c0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:56.634306 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 778/1557] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:56.638702 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b3c62-3999-4834-9123-3124c6881f66 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:56.677618 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:56.707243 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6b3c62-3999-4834-9123-3124c6881f66 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:56.711879 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e8af8-86b2-41b8-8085-2074da18a773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:56.712483 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e8af8-86b2-41b8-8085-2074da18a773 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:56.712765 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e8af8-86b2-41b8-8085-2074da18a773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:56.713040 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97e8af8-86b2-41b8-8085-2074da18a773 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:56.713688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97e8af8-86b2-41b8-8085-2074da18a773 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:56.715624 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97e8af8-86b2-41b8-8085-2074da18a773 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:56.744572 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 780/1558] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:05:56 2022] POST /identity/v3/projects => generated 373 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:56.748718 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73a397-cde3-498b-be16-31966737c0a4 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:56.759980 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73a397-cde3-498b-be16-31966737c0a4 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:56.817285 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628a312-1c4b-482a-91ea-9f78882abe4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:56.817895 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628a312-1c4b-482a-91ea-9f78882abe4b admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:56.818178 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628a312-1c4b-482a-91ea-9f78882abe4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:56.818481 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0628a312-1c4b-482a-91ea-9f78882abe4b admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:56.819408 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0628a312-1c4b-482a-91ea-9f78882abe4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:56.821376 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0628a312-1c4b-482a-91ea-9f78882abe4b admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:56.841318 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 779/1559] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:05:56 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:05:56.845479 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e8af8-86b2-41b8-8085-2074da18a773 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:56.856600 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97e8af8-86b2-41b8-8085-2074da18a773 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:56.861731 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15807d3-c092-41cb-b77b-bf39ad19782d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:56.862398 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15807d3-c092-41cb-b77b-bf39ad19782d admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:56.862654 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15807d3-c092-41cb-b77b-bf39ad19782d admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:56.862933 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15807d3-c092-41cb-b77b-bf39ad19782d admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:56.863585 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15807d3-c092-41cb-b77b-bf39ad19782d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:56.865185 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15807d3-c092-41cb-b77b-bf39ad19782d admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:56.876541 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 781/1560] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:56.880832 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628a312-1c4b-482a-91ea-9f78882abe4b admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:56.891617 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0628a312-1c4b-482a-91ea-9f78882abe4b admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:56.896415 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92780aee-4a07-477e-b3a8-8f2f0f001a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:56.897090 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92780aee-4a07-477e-b3a8-8f2f0f001a41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:56.897889 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92780aee-4a07-477e-b3a8-8f2f0f001a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:56.898166 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92780aee-4a07-477e-b3a8-8f2f0f001a41 admin admin] PATH_INFO: `/v3/projects/85aebb6b076b40baa59daa51240308a5/users/90244645672c416083fd9f1ceaad57f3/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:56.910446 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:05:56.915680 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92780aee-4a07-477e-b3a8-8f2f0f001a41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85aebb6b076b40baa59daa51240308a5, user_id=90244645672c416083fd9f1ceaad57f3, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:56.924022 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92780aee-4a07-477e-b3a8-8f2f0f001a41 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:56.945577 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 780/1561] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:56 2022] PUT /identity/v3/projects/85aebb6b076b40baa59daa51240308a5/users/90244645672c416083fd9f1ceaad57f3/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:56.949065 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15807d3-c092-41cb-b77b-bf39ad19782d admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:56.959131 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15807d3-c092-41cb-b77b-bf39ad19782d admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:57.022558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085e4fa4-e125-41e9-ae3a-57e5165fa7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:57.023181 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e4fa4-e125-41e9-ae3a-57e5165fa7d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:57.023460 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e4fa4-e125-41e9-ae3a-57e5165fa7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:57.023729 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085e4fa4-e125-41e9-ae3a-57e5165fa7d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:57.024402 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085e4fa4-e125-41e9-ae3a-57e5165fa7d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:57.026034 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085e4fa4-e125-41e9-ae3a-57e5165fa7d4 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:57.035910 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 782/1562] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:05:56 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:05:57.040101 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92780aee-4a07-477e-b3a8-8f2f0f001a41 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:57.050422 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92780aee-4a07-477e-b3a8-8f2f0f001a41 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:57.055512 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e945a3d-4de0-410c-98c8-ef326c7f67ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:57.056158 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e945a3d-4de0-410c-98c8-ef326c7f67ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:57.056437 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e945a3d-4de0-410c-98c8-ef326c7f67ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:57.056712 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e945a3d-4de0-410c-98c8-ef326c7f67ba admin admin] PATH_INFO: `/v3/projects/85aebb6b076b40baa59daa51240308a5/users/90244645672c416083fd9f1ceaad57f3/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:57.058664 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e945a3d-4de0-410c-98c8-ef326c7f67ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=85aebb6b076b40baa59daa51240308a5, user_id=90244645672c416083fd9f1ceaad57f3, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:57.061291 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e945a3d-4de0-410c-98c8-ef326c7f67ba admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:57.072247 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 781/1563] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:05:57 2022] PUT /identity/v3/projects/85aebb6b076b40baa59daa51240308a5/users/90244645672c416083fd9f1ceaad57f3/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:05:57.083054 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f92adb-0001-42bd-a8e7-89171ba21958 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:57.083339 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f92adb-0001-42bd-a8e7-89171ba21958 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:57.083610 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f92adb-0001-42bd-a8e7-89171ba21958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:57.110360 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-c5f92adb-0001-42bd-a8e7-89171ba21958 None None] MFA Rules not processed for user `90244645672c416083fd9f1ceaad57f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:05:57.111545 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c5f92adb-0001-42bd-a8e7-89171ba21958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:57.180496 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 783/1564] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:05:57 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:05:57.191014 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e945a3d-4de0-410c-98c8-ef326c7f67ba admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:05:57.192733 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-9e945a3d-4de0-410c-98c8-ef326c7f67ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:05:57.253493 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e945a3d-4de0-410c-98c8-ef326c7f67ba admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:05:57.257727 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953eed12-f2be-403d-a43f-e87a699e6226 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:05:57.258372 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953eed12-f2be-403d-a43f-e87a699e6226 service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:05:57.258651 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953eed12-f2be-403d-a43f-e87a699e6226 service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:05:57.258921 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953eed12-f2be-403d-a43f-e87a699e6226 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:05:57.316161 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953eed12-f2be-403d-a43f-e87a699e6226 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:05:57.317593 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953eed12-f2be-403d-a43f-e87a699e6226 service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:05:57.327902 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 782/1565] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:05:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:06:07.109091 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa7979-f856-4a2e-8ae0-16db70aa287b None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:07.109638 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa7979-f856-4a2e-8ae0-16db70aa287b None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:07.109819 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aa7979-f856-4a2e-8ae0-16db70aa287b None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:07.111896 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 784/1566] 10.222.0.13 () {54 vars in 921 bytes} [Sun Oct 23 20:06:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:06:07.127103 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e909b-c3bc-4bc2-a339-b921dc659c04 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:07.127103 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e909b-c3bc-4bc2-a339-b921dc659c04 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:07.127103 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e909b-c3bc-4bc2-a339-b921dc659c04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:07.153000 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-3e4e909b-c3bc-4bc2-a339-b921dc659c04 None None] MFA Rules not processed for user `0b3d512304254ffcaa8dad90abf1f9c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:07.154631 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-3e4e909b-c3bc-4bc2-a339-b921dc659c04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:07.179248 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=87,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:07.240090 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 783/1567] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Oct 23 20:06:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:07.252758 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa7979-f856-4a2e-8ae0-16db70aa287b None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:07.254559 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-81aa7979-f856-4a2e-8ae0-16db70aa287b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:07.271945 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aa7979-f856-4a2e-8ae0-16db70aa287b None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:07.278433 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bea9101-f17e-4637-b986-1aa6741eab9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:07.279091 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea9101-f17e-4637-b986-1aa6741eab9a service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:07.279432 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea9101-f17e-4637-b986-1aa6741eab9a service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:07.279710 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bea9101-f17e-4637-b986-1aa6741eab9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:07.289934 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea9101-f17e-4637-b986-1aa6741eab9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:07.291574 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bea9101-f17e-4637-b986-1aa6741eab9a service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:07.304790 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 785/1568] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:06:07 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:06:10.507186 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e909b-c3bc-4bc2-a339-b921dc659c04 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:10.531890 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:10.570017 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e909b-c3bc-4bc2-a339-b921dc659c04 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:10.574668 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd46de8-fa1e-40c8-a384-8bf37cb0dd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:10.575301 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd46de8-fa1e-40c8-a384-8bf37cb0dd1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:10.575584 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd46de8-fa1e-40c8-a384-8bf37cb0dd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:10.575860 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd46de8-fa1e-40c8-a384-8bf37cb0dd1e admin admin] PATH_INFO: `/v3/users/f8fc5e88de3549fb860fa99f5adb0cb0` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:10.576980 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd46de8-fa1e-40c8-a384-8bf37cb0dd1e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8fc5e88de3549fb860fa99f5adb0cb0)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:10.578945 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd46de8-fa1e-40c8-a384-8bf37cb0dd1e admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:10.623704 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-8dd46de8-fa1e-40c8-a384-8bf37cb0dd1e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8fc5e88de3549fb860fa99f5adb0cb0'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:10.636074 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-8dd46de8-fa1e-40c8-a384-8bf37cb0dd1e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8fc5e88de3549fb860fa99f5adb0cb0'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:10.642857 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-8dd46de8-fa1e-40c8-a384-8bf37cb0dd1e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8fc5e88de3549fb860fa99f5adb0cb0'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:10.650508 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-8dd46de8-fa1e-40c8-a384-8bf37cb0dd1e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8fc5e88de3549fb860fa99f5adb0cb0'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:10.651154 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-8dd46de8-fa1e-40c8-a384-8bf37cb0dd1e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8fc5e88de3549fb860fa99f5adb0cb0'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:10.657701 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 784/1569] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:06:10 2022] DELETE /identity/v3/users/f8fc5e88de3549fb860fa99f5adb0cb0 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:10.662169 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6476b-e1d4-40e9-9376-bb15a6e26d45 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:10.662769 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6476b-e1d4-40e9-9376-bb15a6e26d45 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:10.662769 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d6476b-e1d4-40e9-9376-bb15a6e26d45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:10.683062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-c0d6476b-e1d4-40e9-9376-bb15a6e26d45 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:10.684274 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c0d6476b-e1d4-40e9-9376-bb15a6e26d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:10.751301 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 786/1570] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:06:10.761626 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd46de8-fa1e-40c8-a384-8bf37cb0dd1e admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:10.763288 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-8dd46de8-fa1e-40c8-a384-8bf37cb0dd1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:10.842731 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd46de8-fa1e-40c8-a384-8bf37cb0dd1e admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:10.853448 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d3d6b4-0cb0-40f0-9447-46c2082fe978 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:10.854260 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3d6b4-0cb0-40f0-9447-46c2082fe978 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:10.854558 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3d6b4-0cb0-40f0-9447-46c2082fe978 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:10.854837 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d3d6b4-0cb0-40f0-9447-46c2082fe978 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:10.868737 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d3d6b4-0cb0-40f0-9447-46c2082fe978 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:10.870352 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d3d6b4-0cb0-40f0-9447-46c2082fe978 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:10.883459 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 785/1571] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:06:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:06:10.997546 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d6476b-e1d4-40e9-9376-bb15a6e26d45 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:10.999069 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c0d6476b-e1d4-40e9-9376-bb15a6e26d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:11.011888 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d6476b-e1d4-40e9-9376-bb15a6e26d45 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:11.023626 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba074a-e304-4e05-b13f-4ba07ba72724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:11.024245 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba074a-e304-4e05-b13f-4ba07ba72724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:11.024526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba074a-e304-4e05-b13f-4ba07ba72724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:11.024801 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba074a-e304-4e05-b13f-4ba07ba72724 admin admin] PATH_INFO: `/v3/users/6e10299fabad4dc6895a65631025584b` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:11.025977 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba074a-e304-4e05-b13f-4ba07ba72724 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e10299fabad4dc6895a65631025584b)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:11.027927 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba074a-e304-4e05-b13f-4ba07ba72724 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:11.079071 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-2cba074a-e304-4e05-b13f-4ba07ba72724 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e10299fabad4dc6895a65631025584b'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:11.093491 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-2cba074a-e304-4e05-b13f-4ba07ba72724 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e10299fabad4dc6895a65631025584b'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:11.103458 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-2cba074a-e304-4e05-b13f-4ba07ba72724 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e10299fabad4dc6895a65631025584b'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:11.104631 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-2cba074a-e304-4e05-b13f-4ba07ba72724 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e10299fabad4dc6895a65631025584b'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:11.112621 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-2cba074a-e304-4e05-b13f-4ba07ba72724 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e10299fabad4dc6895a65631025584b'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:11.119021 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 787/1572] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:06:10 2022] DELETE /identity/v3/users/6e10299fabad4dc6895a65631025584b => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:11.123462 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff475da-e1da-4388-8fff-b59a7b0e108c None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:11.123820 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff475da-e1da-4388-8fff-b59a7b0e108c None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:11.124163 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff475da-e1da-4388-8fff-b59a7b0e108c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:11.143321 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-6ff475da-e1da-4388-8fff-b59a7b0e108c None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:11.144537 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-6ff475da-e1da-4388-8fff-b59a7b0e108c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:11.189541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:11.189877 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba074a-e304-4e05-b13f-4ba07ba72724 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:11.191348 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2cba074a-e304-4e05-b13f-4ba07ba72724 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:11.212953 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 786/1573] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:11.223446 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff475da-e1da-4388-8fff-b59a7b0e108c None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:11.224923 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-6ff475da-e1da-4388-8fff-b59a7b0e108c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:11.267700 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba074a-e304-4e05-b13f-4ba07ba72724 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:11.273631 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1e7b5-bb2a-4027-8091-79de89580c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:11.274306 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1e7b5-bb2a-4027-8091-79de89580c51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:11.274588 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1e7b5-bb2a-4027-8091-79de89580c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:11.274868 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d1e7b5-bb2a-4027-8091-79de89580c51 admin admin] PATH_INFO: `/v3/projects/6ac0b628da0b4de1bac868f0cfd39236` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:11.276000 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1e7b5-bb2a-4027-8091-79de89580c51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ac0b628da0b4de1bac868f0cfd39236)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:11.277906 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d1e7b5-bb2a-4027-8091-79de89580c51 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:11.279608 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:11.284423 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff475da-e1da-4388-8fff-b59a7b0e108c None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:11.288705 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f3a31-73f3-4a71-810d-4f6898d86a91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:11.289309 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f3a31-73f3-4a71-810d-4f6898d86a91 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:11.289652 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f3a31-73f3-4a71-810d-4f6898d86a91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:11.289854 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f3a31-73f3-4a71-810d-4f6898d86a91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:11.299188 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f3a31-73f3-4a71-810d-4f6898d86a91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:11.300616 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f3a31-73f3-4a71-810d-4f6898d86a91 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:11.310616 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 787/1574] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:06:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:06:11.327376 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f1d1e7b5-bb2a-4027-8091-79de89580c51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ac0b628da0b4de1bac868f0cfd39236', 'request_id': 'req-f1d1e7b5-bb2a-4027-8091-79de89580c51'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:11.336466 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f1d1e7b5-bb2a-4027-8091-79de89580c51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ac0b628da0b4de1bac868f0cfd39236', 'request_id': 'req-f1d1e7b5-bb2a-4027-8091-79de89580c51'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:11.342669 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f1d1e7b5-bb2a-4027-8091-79de89580c51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ac0b628da0b4de1bac868f0cfd39236', 'request_id': 'req-f1d1e7b5-bb2a-4027-8091-79de89580c51'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:11.354793 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f1d1e7b5-bb2a-4027-8091-79de89580c51 admin admin] The token cache is being invalidate because project 6ac0b628da0b4de1bac868f0cfd39236 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:06:11.355086 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f1d1e7b5-bb2a-4027-8091-79de89580c51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:11.355877 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f1d1e7b5-bb2a-4027-8091-79de89580c51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:11.357542 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 788/1575] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:06:11 2022] DELETE /identity/v3/projects/6ac0b628da0b4de1bac868f0cfd39236 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:06:11.361600 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f3a31-73f3-4a71-810d-4f6898d86a91 service neutron] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:11.374459 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f3a31-73f3-4a71-810d-4f6898d86a91 service neutron] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:11.449054 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c8a470-276d-45ef-8124-5134f74c75df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:11.449699 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8a470-276d-45ef-8124-5134f74c75df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:11.449980 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8a470-276d-45ef-8124-5134f74c75df admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:11.450306 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c8a470-276d-45ef-8124-5134f74c75df admin admin] PATH_INFO: `/v3/users/90244645672c416083fd9f1ceaad57f3` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:11.451427 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c8a470-276d-45ef-8124-5134f74c75df admin admin] RBAC: Authorizing `identity:delete_user(user_id=90244645672c416083fd9f1ceaad57f3)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:11.453496 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c8a470-276d-45ef-8124-5134f74c75df admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:11.503165 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-24c8a470-276d-45ef-8124-5134f74c75df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90244645672c416083fd9f1ceaad57f3'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:11.518318 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-24c8a470-276d-45ef-8124-5134f74c75df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90244645672c416083fd9f1ceaad57f3'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:11.527201 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-24c8a470-276d-45ef-8124-5134f74c75df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90244645672c416083fd9f1ceaad57f3'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:11.537518 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-24c8a470-276d-45ef-8124-5134f74c75df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90244645672c416083fd9f1ceaad57f3'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:11.538275 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-24c8a470-276d-45ef-8124-5134f74c75df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90244645672c416083fd9f1ceaad57f3'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:11.546226 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 788/1576] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:06:11 2022] DELETE /identity/v3/users/90244645672c416083fd9f1ceaad57f3 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:11.835839 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1e7b5-bb2a-4027-8091-79de89580c51 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:11.838606 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f1d1e7b5-bb2a-4027-8091-79de89580c51 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:11.853540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d1e7b5-bb2a-4027-8091-79de89580c51 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:11.918317 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c05e68-1fd8-496f-a0b0-880b1991c576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:11.918963 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c05e68-1fd8-496f-a0b0-880b1991c576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:11.919386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c05e68-1fd8-496f-a0b0-880b1991c576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:11.919673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c05e68-1fd8-496f-a0b0-880b1991c576 admin admin] PATH_INFO: `/v3/projects/ef0abe2425994dcab2d2e2368a0de717` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:11.920821 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c05e68-1fd8-496f-a0b0-880b1991c576 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef0abe2425994dcab2d2e2368a0de717)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:11.922873 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c05e68-1fd8-496f-a0b0-880b1991c576 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:11.947052 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c8a470-276d-45ef-8124-5134f74c75df admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:11.948640 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-24c8a470-276d-45ef-8124-5134f74c75df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:11.964831 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c8a470-276d-45ef-8124-5134f74c75df admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:11.971831 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c282ce-3710-4cdb-badd-3e6b3eaf9ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:11.972790 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c282ce-3710-4cdb-badd-3e6b3eaf9ae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:11.973084 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c282ce-3710-4cdb-badd-3e6b3eaf9ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:11.973332 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c282ce-3710-4cdb-badd-3e6b3eaf9ae8 admin admin] PATH_INFO: `/v3/projects/85aebb6b076b40baa59daa51240308a5` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:11.974678 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c282ce-3710-4cdb-badd-3e6b3eaf9ae8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85aebb6b076b40baa59daa51240308a5)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:11.976631 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c282ce-3710-4cdb-badd-3e6b3eaf9ae8 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:11.984196 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:11.998950 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-c3c05e68-1fd8-496f-a0b0-880b1991c576 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef0abe2425994dcab2d2e2368a0de717', 'request_id': 'req-c3c05e68-1fd8-496f-a0b0-880b1991c576'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:12.022147 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-c3c05e68-1fd8-496f-a0b0-880b1991c576 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef0abe2425994dcab2d2e2368a0de717', 'request_id': 'req-c3c05e68-1fd8-496f-a0b0-880b1991c576'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:12.029980 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-c3c05e68-1fd8-496f-a0b0-880b1991c576 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef0abe2425994dcab2d2e2368a0de717', 'request_id': 'req-c3c05e68-1fd8-496f-a0b0-880b1991c576'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:12.044108 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-c3c05e68-1fd8-496f-a0b0-880b1991c576 admin admin] The token cache is being invalidate because project ef0abe2425994dcab2d2e2368a0de717 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:06:12.044462 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-c3c05e68-1fd8-496f-a0b0-880b1991c576 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:12.045544 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-c3c05e68-1fd8-496f-a0b0-880b1991c576 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:12.047122 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 789/1577] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:06:11 2022] DELETE /identity/v3/projects/ef0abe2425994dcab2d2e2368a0de717 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:12.051972 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c05e68-1fd8-496f-a0b0-880b1991c576 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:12.055746 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c3c05e68-1fd8-496f-a0b0-880b1991c576 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:12.068325 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-f3c282ce-3710-4cdb-badd-3e6b3eaf9ae8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85aebb6b076b40baa59daa51240308a5', 'request_id': 'req-f3c282ce-3710-4cdb-badd-3e6b3eaf9ae8'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:12.078558 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-f3c282ce-3710-4cdb-badd-3e6b3eaf9ae8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85aebb6b076b40baa59daa51240308a5', 'request_id': 'req-f3c282ce-3710-4cdb-badd-3e6b3eaf9ae8'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:12.080083 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:12.085987 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-f3c282ce-3710-4cdb-badd-3e6b3eaf9ae8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85aebb6b076b40baa59daa51240308a5', 'request_id': 'req-f3c282ce-3710-4cdb-badd-3e6b3eaf9ae8'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:12.104965 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-f3c282ce-3710-4cdb-badd-3e6b3eaf9ae8 admin admin] The token cache is being invalidate because project 85aebb6b076b40baa59daa51240308a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:06:12.105650 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-f3c282ce-3710-4cdb-badd-3e6b3eaf9ae8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:12.106702 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-f3c282ce-3710-4cdb-badd-3e6b3eaf9ae8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:12.109104 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 789/1578] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:06:11 2022] DELETE /identity/v3/projects/85aebb6b076b40baa59daa51240308a5 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:06:12.114288 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bec21a-4df5-4c52-a9e4-f378d00216d5 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:12.114678 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bec21a-4df5-4c52-a9e4-f378d00216d5 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:12.114928 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bec21a-4df5-4c52-a9e4-f378d00216d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:12.142579 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-e5bec21a-4df5-4c52-a9e4-f378d00216d5 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:12.143814 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-e5bec21a-4df5-4c52-a9e4-f378d00216d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:12.151913 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c05e68-1fd8-496f-a0b0-880b1991c576 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:12.209364 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 790/1579] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:06:12.226088 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71445c-9e91-4f5c-afa7-b66791466854 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:12.226088 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71445c-9e91-4f5c-afa7-b66791466854 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:12.226088 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f71445c-9e91-4f5c-afa7-b66791466854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:12.242056 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283dbfc-e416-4626-a590-f47cdf46bcb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:12.242747 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283dbfc-e416-4626-a590-f47cdf46bcb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:12.243035 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283dbfc-e416-4626-a590-f47cdf46bcb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:12.243313 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283dbfc-e416-4626-a590-f47cdf46bcb2 admin admin] PATH_INFO: `/v3/users/1b08c2d95a8e447490d4f94db18d3c3d` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:12.244475 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5283dbfc-e416-4626-a590-f47cdf46bcb2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b08c2d95a8e447490d4f94db18d3c3d)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:12.246483 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5283dbfc-e416-4626-a590-f47cdf46bcb2 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:12.249829 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-4f71445c-9e91-4f5c-afa7-b66791466854 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:12.251152 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-4f71445c-9e91-4f5c-afa7-b66791466854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:12.259265 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 791/1580] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:06:12.263377 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f71445c-9e91-4f5c-afa7-b66791466854 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:12.273801 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f71445c-9e91-4f5c-afa7-b66791466854 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:12.278957 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba3a20-26fe-4197-b8c1-a18fb9f30d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:12.279579 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba3a20-26fe-4197-b8c1-a18fb9f30d49 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:12.279860 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba3a20-26fe-4197-b8c1-a18fb9f30d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:12.280140 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eba3a20-26fe-4197-b8c1-a18fb9f30d49 admin admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:12.280874 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba3a20-26fe-4197-b8c1-a18fb9f30d49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:06:12.281185 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba3a20-26fe-4197-b8c1-a18fb9f30d49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:12.282828 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eba3a20-26fe-4197-b8c1-a18fb9f30d49 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:12.293929 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5283dbfc-e416-4626-a590-f47cdf46bcb2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b08c2d95a8e447490d4f94db18d3c3d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:12.295473 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 792/1581] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 20:06:12.299792 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba3a20-26fe-4197-b8c1-a18fb9f30d49 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:12.307247 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5283dbfc-e416-4626-a590-f47cdf46bcb2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b08c2d95a8e447490d4f94db18d3c3d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:12.312020 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eba3a20-26fe-4197-b8c1-a18fb9f30d49 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:12.316424 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5283dbfc-e416-4626-a590-f47cdf46bcb2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b08c2d95a8e447490d4f94db18d3c3d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:12.317117 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5283dbfc-e416-4626-a590-f47cdf46bcb2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b08c2d95a8e447490d4f94db18d3c3d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:12.317311 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa4c2f-dfc4-4afb-8d98-4f9a9db0e430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:12.318057 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa4c2f-dfc4-4afb-8d98-4f9a9db0e430 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:12.318373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa4c2f-dfc4-4afb-8d98-4f9a9db0e430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:12.318678 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5aa4c2f-dfc4-4afb-8d98-4f9a9db0e430 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:12.319597 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa4c2f-dfc4-4afb-8d98-4f9a9db0e430 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:12.321477 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5aa4c2f-dfc4-4afb-8d98-4f9a9db0e430 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:12.324830 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5283dbfc-e416-4626-a590-f47cdf46bcb2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b08c2d95a8e447490d4f94db18d3c3d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:12.331900 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 790/1582] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:06:12 2022] DELETE /identity/v3/users/1b08c2d95a8e447490d4f94db18d3c3d => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:06:12.352802 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 793/1583] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:06:12 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:06:12.356664 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283dbfc-e416-4626-a590-f47cdf46bcb2 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:12.358574 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5283dbfc-e416-4626-a590-f47cdf46bcb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:12.410015 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:12.429117 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283dbfc-e416-4626-a590-f47cdf46bcb2 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:12.495652 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd754bd-a65c-461a-88d1-f19a1e352b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:12.496274 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd754bd-a65c-461a-88d1-f19a1e352b63 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:12.496556 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd754bd-a65c-461a-88d1-f19a1e352b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:12.496828 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd754bd-a65c-461a-88d1-f19a1e352b63 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:12.497480 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd754bd-a65c-461a-88d1-f19a1e352b63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:12.499430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd754bd-a65c-461a-88d1-f19a1e352b63 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:12.520927 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 791/1584] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:06:12 2022] POST /identity/v3/users => generated 288 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:06:12.524919 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa4c2f-dfc4-4afb-8d98-4f9a9db0e430 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:12.535628 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5aa4c2f-dfc4-4afb-8d98-4f9a9db0e430 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:12.540831 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d0ef9-800f-49a5-b7ee-92a3512a70da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:12.541486 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d0ef9-800f-49a5-b7ee-92a3512a70da admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:12.541848 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d0ef9-800f-49a5-b7ee-92a3512a70da admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:12.542266 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377d0ef9-800f-49a5-b7ee-92a3512a70da admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:12.542922 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d0ef9-800f-49a5-b7ee-92a3512a70da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:12.544545 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377d0ef9-800f-49a5-b7ee-92a3512a70da admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:12.558414 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 794/1585] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:06:12.559849 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd754bd-a65c-461a-88d1-f19a1e352b63 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:12.570926 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd754bd-a65c-461a-88d1-f19a1e352b63 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:12.576381 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16591aab-8d3f-4167-b5d3-11120b44456c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:12.577113 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16591aab-8d3f-4167-b5d3-11120b44456c admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:12.577483 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16591aab-8d3f-4167-b5d3-11120b44456c admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:12.577803 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16591aab-8d3f-4167-b5d3-11120b44456c admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:12.578569 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16591aab-8d3f-4167-b5d3-11120b44456c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:12.580483 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16591aab-8d3f-4167-b5d3-11120b44456c admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:12.592727 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 792/1586] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:12 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:06:12.597373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d0ef9-800f-49a5-b7ee-92a3512a70da admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:12.609294 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377d0ef9-800f-49a5-b7ee-92a3512a70da admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:12.615662 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9feb20-2a04-4756-86fe-b1bf417672eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:12.616467 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9feb20-2a04-4756-86fe-b1bf417672eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:12.616812 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9feb20-2a04-4756-86fe-b1bf417672eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:12.617163 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9feb20-2a04-4756-86fe-b1bf417672eb admin admin] PATH_INFO: `/v3/projects/4182ccd297e54d4b8fad97db74777b7c/users/dc82bd89cd4d407a85fe59a5fee5f2b0/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:12.619393 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16591aab-8d3f-4167-b5d3-11120b44456c admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:12.622818 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-16591aab-8d3f-4167-b5d3-11120b44456c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:12.640086 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16591aab-8d3f-4167-b5d3-11120b44456c admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:12.654271 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9feb20-2a04-4756-86fe-b1bf417672eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4182ccd297e54d4b8fad97db74777b7c, user_id=dc82bd89cd4d407a85fe59a5fee5f2b0, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:12.654608 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b403eeb-47f7-4433-9bf3-c6d4ad2d5f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:12.654608 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b403eeb-47f7-4433-9bf3-c6d4ad2d5f8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:12.654608 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b403eeb-47f7-4433-9bf3-c6d4ad2d5f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:12.654608 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b403eeb-47f7-4433-9bf3-c6d4ad2d5f8f admin admin] PATH_INFO: `/v3/projects/05889dc8180c49eba24edc7510cd9714` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:12.654888 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b403eeb-47f7-4433-9bf3-c6d4ad2d5f8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=05889dc8180c49eba24edc7510cd9714)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:12.654966 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9feb20-2a04-4756-86fe-b1bf417672eb admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:12.655036 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b403eeb-47f7-4433-9bf3-c6d4ad2d5f8f admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:12.668295 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 795/1587] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:06:12 2022] PUT /identity/v3/projects/4182ccd297e54d4b8fad97db74777b7c/users/dc82bd89cd4d407a85fe59a5fee5f2b0/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:12.679515 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7522d4-9b1b-46b6-bf46-07d57c9b213a None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:12.679768 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7522d4-9b1b-46b6-bf46-07d57c9b213a None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:12.680048 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7522d4-9b1b-46b6-bf46-07d57c9b213a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:12.708738 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:12.712891 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-0c7522d4-9b1b-46b6-bf46-07d57c9b213a None None] MFA Rules not processed for user `dc82bd89cd4d407a85fe59a5fee5f2b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:12.714460 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-0c7522d4-9b1b-46b6-bf46-07d57c9b213a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:12.715778 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-2b403eeb-47f7-4433-9bf3-c6d4ad2d5f8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05889dc8180c49eba24edc7510cd9714', 'request_id': 'req-2b403eeb-47f7-4433-9bf3-c6d4ad2d5f8f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:12.728284 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-2b403eeb-47f7-4433-9bf3-c6d4ad2d5f8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05889dc8180c49eba24edc7510cd9714', 'request_id': 'req-2b403eeb-47f7-4433-9bf3-c6d4ad2d5f8f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:12.743781 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-2b403eeb-47f7-4433-9bf3-c6d4ad2d5f8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05889dc8180c49eba24edc7510cd9714', 'request_id': 'req-2b403eeb-47f7-4433-9bf3-c6d4ad2d5f8f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:12.771698 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-2b403eeb-47f7-4433-9bf3-c6d4ad2d5f8f admin admin] The token cache is being invalidate because project 05889dc8180c49eba24edc7510cd9714 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:06:12.772087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-2b403eeb-47f7-4433-9bf3-c6d4ad2d5f8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:12.772724 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-2b403eeb-47f7-4433-9bf3-c6d4ad2d5f8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:12.774751 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 793/1588] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:06:12 2022] DELETE /identity/v3/projects/05889dc8180c49eba24edc7510cd9714 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:12.778983 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747fac4-f7fe-4cff-8fab-eb6175c3eb34 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:12.779489 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747fac4-f7fe-4cff-8fab-eb6175c3eb34 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:12.780040 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9747fac4-f7fe-4cff-8fab-eb6175c3eb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:12.807034 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-9747fac4-f7fe-4cff-8fab-eb6175c3eb34 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:12.808472 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9747fac4-f7fe-4cff-8fab-eb6175c3eb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:12.832921 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 796/1589] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:06:12.849444 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7522d4-9b1b-46b6-bf46-07d57c9b213a None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:12.851082 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-0c7522d4-9b1b-46b6-bf46-07d57c9b213a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:12.895096 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:12.914866 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 794/1590] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:06:12.922709 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c014a-d973-418f-b591-5fe33ea1ff19 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:12.923531 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c014a-d973-418f-b591-5fe33ea1ff19 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:12.923843 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154c014a-d973-418f-b591-5fe33ea1ff19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:12.946640 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7522d4-9b1b-46b6-bf46-07d57c9b213a None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:12.955325 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a97e393-d32d-4599-b2b9-3659d2b1bf71 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:12.956010 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97e393-d32d-4599-b2b9-3659d2b1bf71 service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:12.956303 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97e393-d32d-4599-b2b9-3659d2b1bf71 service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:12.956606 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-154c014a-d973-418f-b591-5fe33ea1ff19 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:12.956714 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97e393-d32d-4599-b2b9-3659d2b1bf71 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:12.958569 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-154c014a-d973-418f-b591-5fe33ea1ff19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:12.969925 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a97e393-d32d-4599-b2b9-3659d2b1bf71 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:12.971520 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a97e393-d32d-4599-b2b9-3659d2b1bf71 service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:12.974632 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 795/1591] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:12.976284 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c014a-d973-418f-b591-5fe33ea1ff19 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:12.985201 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 797/1592] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:06:12 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 20:06:13.005851 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154c014a-d973-418f-b591-5fe33ea1ff19 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:13.005851 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dba4c7-4474-4631-9fb5-c97a84a62249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:13.005851 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dba4c7-4474-4631-9fb5-c97a84a62249 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:13.005851 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dba4c7-4474-4631-9fb5-c97a84a62249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:13.006919 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dba4c7-4474-4631-9fb5-c97a84a62249 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:13.006919 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dba4c7-4474-4631-9fb5-c97a84a62249 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:06:13.006919 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dba4c7-4474-4631-9fb5-c97a84a62249 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:13.008000 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dba4c7-4474-4631-9fb5-c97a84a62249 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:13.026292 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 796/1593] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:06:13.030601 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a97e393-d32d-4599-b2b9-3659d2b1bf71 service cinder] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:13.041460 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a97e393-d32d-4599-b2b9-3659d2b1bf71 service cinder] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:13.046128 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc7222-4e32-4f00-94db-ac3c1689c46f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:13.046759 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc7222-4e32-4f00-94db-ac3c1689c46f admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:13.047050 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc7222-4e32-4f00-94db-ac3c1689c46f admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:13.047323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bc7222-4e32-4f00-94db-ac3c1689c46f admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:13.048014 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc7222-4e32-4f00-94db-ac3c1689c46f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:13.049935 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bc7222-4e32-4f00-94db-ac3c1689c46f admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:13.079037 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 798/1594] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:06:13 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:06:13.083240 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dba4c7-4474-4631-9fb5-c97a84a62249 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:13.093838 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dba4c7-4474-4631-9fb5-c97a84a62249 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:13.158095 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a848200-cda0-44cb-9d02-2d6823839616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:13.158712 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a848200-cda0-44cb-9d02-2d6823839616 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:13.159020 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a848200-cda0-44cb-9d02-2d6823839616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:13.159292 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a848200-cda0-44cb-9d02-2d6823839616 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:13.159929 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a848200-cda0-44cb-9d02-2d6823839616 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:13.161840 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a848200-cda0-44cb-9d02-2d6823839616 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:13.182438 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 797/1595] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:06:13 2022] POST /identity/v3/users => generated 283 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:06:13.186591 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc7222-4e32-4f00-94db-ac3c1689c46f admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:13.197824 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bc7222-4e32-4f00-94db-ac3c1689c46f admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:13.203544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27753d87-dda8-498f-a2e4-b4723a59251d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:13.204264 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27753d87-dda8-498f-a2e4-b4723a59251d admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:13.204443 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27753d87-dda8-498f-a2e4-b4723a59251d admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:13.204844 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27753d87-dda8-498f-a2e4-b4723a59251d admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:13.205371 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27753d87-dda8-498f-a2e4-b4723a59251d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:13.207087 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27753d87-dda8-498f-a2e4-b4723a59251d admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:13.217345 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 799/1596] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:06:13.221611 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a848200-cda0-44cb-9d02-2d6823839616 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:13.242778 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a848200-cda0-44cb-9d02-2d6823839616 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:13.242778 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc34ba05-e38c-4fa2-b05e-af977eaf7abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:13.242778 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34ba05-e38c-4fa2-b05e-af977eaf7abf admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:13.242778 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34ba05-e38c-4fa2-b05e-af977eaf7abf admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:13.243050 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc34ba05-e38c-4fa2-b05e-af977eaf7abf admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:13.243050 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc34ba05-e38c-4fa2-b05e-af977eaf7abf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:13.248108 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc34ba05-e38c-4fa2-b05e-af977eaf7abf admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:13.279944 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 798/1597] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:06:13.284509 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27753d87-dda8-498f-a2e4-b4723a59251d admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:13.304005 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27753d87-dda8-498f-a2e4-b4723a59251d admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:13.312308 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6522487-e368-41d5-9012-82209784668b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:13.313971 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6522487-e368-41d5-9012-82209784668b admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:13.314897 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6522487-e368-41d5-9012-82209784668b admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:13.315772 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6522487-e368-41d5-9012-82209784668b admin admin] PATH_INFO: `/v3/projects/8ecc025a0a434b69863a51807c7fa078/users/072140f112234d5295fead08bde4185c/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:13.334574 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6522487-e368-41d5-9012-82209784668b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ecc025a0a434b69863a51807c7fa078, user_id=072140f112234d5295fead08bde4185c, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:13.337543 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6522487-e368-41d5-9012-82209784668b admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:13.356467 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 800/1598] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:06:13 2022] PUT /identity/v3/projects/8ecc025a0a434b69863a51807c7fa078/users/072140f112234d5295fead08bde4185c/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:13.368196 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7fefd8-f1e2-4c73-993d-ebbf8fc1d422 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:13.368513 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7fefd8-f1e2-4c73-993d-ebbf8fc1d422 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:13.368789 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7fefd8-f1e2-4c73-993d-ebbf8fc1d422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:13.398917 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-ff7fefd8-f1e2-4c73-993d-ebbf8fc1d422 None None] MFA Rules not processed for user `072140f112234d5295fead08bde4185c`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:13.400162 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ff7fefd8-f1e2-4c73-993d-ebbf8fc1d422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:13.469121 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 799/1599] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:13.475960 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6522487-e368-41d5-9012-82209784668b admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:13.540160 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:13.554795 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6522487-e368-41d5-9012-82209784668b admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:13.559436 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1fa0bc-66ad-40ce-9704-bf8bf2028735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:13.560044 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fa0bc-66ad-40ce-9704-bf8bf2028735 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:13.560327 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fa0bc-66ad-40ce-9704-bf8bf2028735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:13.560601 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fa0bc-66ad-40ce-9704-bf8bf2028735 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:13.561231 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1fa0bc-66ad-40ce-9704-bf8bf2028735 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:13.563217 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1fa0bc-66ad-40ce-9704-bf8bf2028735 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:13.588867 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 801/1600] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:06:13 2022] POST /identity/v3/projects => generated 365 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:06:13.593032 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7fefd8-f1e2-4c73-993d-ebbf8fc1d422 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:13.603696 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7fefd8-f1e2-4c73-993d-ebbf8fc1d422 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:13.645096 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:13.661366 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5fe584-f1c9-415e-929c-e8bf285ece96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:13.662114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5fe584-f1c9-415e-929c-e8bf285ece96 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:13.662204 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5fe584-f1c9-415e-929c-e8bf285ece96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:13.662598 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5fe584-f1c9-415e-929c-e8bf285ece96 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:13.663156 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5fe584-f1c9-415e-929c-e8bf285ece96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:13.665072 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5fe584-f1c9-415e-929c-e8bf285ece96 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:13.688542 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 800/1601] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:06:13 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:06:13.693080 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1fa0bc-66ad-40ce-9704-bf8bf2028735 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:13.704547 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1fa0bc-66ad-40ce-9704-bf8bf2028735 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:13.709209 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae05d34-1a5b-40cd-9821-0a72b46b1bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:13.709840 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae05d34-1a5b-40cd-9821-0a72b46b1bfe admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:13.710125 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae05d34-1a5b-40cd-9821-0a72b46b1bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:13.710433 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae05d34-1a5b-40cd-9821-0a72b46b1bfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:13.711094 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae05d34-1a5b-40cd-9821-0a72b46b1bfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:13.712776 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae05d34-1a5b-40cd-9821-0a72b46b1bfe admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:13.723389 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 802/1602] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:06:13.728192 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5fe584-f1c9-415e-929c-e8bf285ece96 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:13.738038 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5fe584-f1c9-415e-929c-e8bf285ece96 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:13.743220 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f306b7-7448-4b7d-87d1-0709b9d13bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:13.743875 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f306b7-7448-4b7d-87d1-0709b9d13bed admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:13.744156 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f306b7-7448-4b7d-87d1-0709b9d13bed admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:13.744433 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f306b7-7448-4b7d-87d1-0709b9d13bed admin admin] PATH_INFO: `/v3/projects/6a65d3b182284519bb1c2af56915381a/users/e8eff3a9281744b69ab3ef4e2bc0442f/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:13.759241 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f306b7-7448-4b7d-87d1-0709b9d13bed admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a65d3b182284519bb1c2af56915381a, user_id=e8eff3a9281744b69ab3ef4e2bc0442f, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:13.761894 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f306b7-7448-4b7d-87d1-0709b9d13bed admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:13.775226 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 801/1603] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:06:13 2022] PUT /identity/v3/projects/6a65d3b182284519bb1c2af56915381a/users/e8eff3a9281744b69ab3ef4e2bc0442f/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:13.779133 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae05d34-1a5b-40cd-9821-0a72b46b1bfe admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:13.789791 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae05d34-1a5b-40cd-9821-0a72b46b1bfe admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:13.847623 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a586af4-ee5f-4d33-a42d-a5a62ff20d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:13.848232 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a586af4-ee5f-4d33-a42d-a5a62ff20d60 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:13.848514 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a586af4-ee5f-4d33-a42d-a5a62ff20d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:13.848788 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a586af4-ee5f-4d33-a42d-a5a62ff20d60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:13.849432 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a586af4-ee5f-4d33-a42d-a5a62ff20d60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:13.851583 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a586af4-ee5f-4d33-a42d-a5a62ff20d60 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:13.861241 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 803/1604] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:06:13.865453 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f306b7-7448-4b7d-87d1-0709b9d13bed admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:13.876499 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f306b7-7448-4b7d-87d1-0709b9d13bed admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:13.881163 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15a260-bb14-4a78-a383-ea8f79c4d662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:13.882369 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15a260-bb14-4a78-a383-ea8f79c4d662 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:13.882687 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15a260-bb14-4a78-a383-ea8f79c4d662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:13.882977 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15a260-bb14-4a78-a383-ea8f79c4d662 admin admin] PATH_INFO: `/v3/projects/6a65d3b182284519bb1c2af56915381a/users/e8eff3a9281744b69ab3ef4e2bc0442f/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:13.884909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c15a260-bb14-4a78-a383-ea8f79c4d662 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6a65d3b182284519bb1c2af56915381a, user_id=e8eff3a9281744b69ab3ef4e2bc0442f, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:13.887742 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c15a260-bb14-4a78-a383-ea8f79c4d662 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:13.898654 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 802/1605] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:06:13 2022] PUT /identity/v3/projects/6a65d3b182284519bb1c2af56915381a/users/e8eff3a9281744b69ab3ef4e2bc0442f/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:13.909732 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5312b9-404d-4409-8c62-cdaed542c5dd None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:13.910025 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5312b9-404d-4409-8c62-cdaed542c5dd None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:13.910343 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5312b9-404d-4409-8c62-cdaed542c5dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:13.937745 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-ca5312b9-404d-4409-8c62-cdaed542c5dd None None] MFA Rules not processed for user `e8eff3a9281744b69ab3ef4e2bc0442f`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:13.938924 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-ca5312b9-404d-4409-8c62-cdaed542c5dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:14.007118 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 804/1606] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:13 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:14.024828 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15a260-bb14-4a78-a383-ea8f79c4d662 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:14.096777 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15a260-bb14-4a78-a383-ea8f79c4d662 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:14.100432 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae59bc5-8931-4fc9-be21-64641e3df7d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:14.101032 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae59bc5-8931-4fc9-be21-64641e3df7d1 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:14.101310 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae59bc5-8931-4fc9-be21-64641e3df7d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:14.102160 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae59bc5-8931-4fc9-be21-64641e3df7d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:14.111256 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae59bc5-8931-4fc9-be21-64641e3df7d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:14.112674 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae59bc5-8931-4fc9-be21-64641e3df7d1 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:14.122859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 803/1607] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:06:14 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:06:14.203960 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5312b9-404d-4409-8c62-cdaed542c5dd None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:14.214302 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5312b9-404d-4409-8c62-cdaed542c5dd None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:14.218666 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee39625a-6267-4b84-a1c5-3af167675ac7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:14.219292 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39625a-6267-4b84-a1c5-3af167675ac7 service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:14.219577 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39625a-6267-4b84-a1c5-3af167675ac7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:14.219860 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39625a-6267-4b84-a1c5-3af167675ac7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:14.269732 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:14.286217 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee39625a-6267-4b84-a1c5-3af167675ac7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:14.288238 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee39625a-6267-4b84-a1c5-3af167675ac7 service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:14.297253 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 805/1608] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:06:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:06:16.903667 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae59bc5-8931-4fc9-be21-64641e3df7d1 service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:16.913966 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-aae59bc5-8931-4fc9-be21-64641e3df7d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:16.978450 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae59bc5-8931-4fc9-be21-64641e3df7d1 service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:16.984169 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aed139-c793-4acf-baaa-20ed2e1b9877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:16.984813 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aed139-c793-4acf-baaa-20ed2e1b9877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:16.985091 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aed139-c793-4acf-baaa-20ed2e1b9877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:16.985373 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6aed139-c793-4acf-baaa-20ed2e1b9877 admin admin] PATH_INFO: `/v3/users/3595cfbefb134e27a69dbefa059fc1b1` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:16.986965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aed139-c793-4acf-baaa-20ed2e1b9877 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3595cfbefb134e27a69dbefa059fc1b1)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:16.988863 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6aed139-c793-4acf-baaa-20ed2e1b9877 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:17.035555 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:17.043230 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a6aed139-c793-4acf-baaa-20ed2e1b9877 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3595cfbefb134e27a69dbefa059fc1b1'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:17.060105 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a6aed139-c793-4acf-baaa-20ed2e1b9877 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3595cfbefb134e27a69dbefa059fc1b1'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:17.065789 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a6aed139-c793-4acf-baaa-20ed2e1b9877 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3595cfbefb134e27a69dbefa059fc1b1'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:17.066557 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a6aed139-c793-4acf-baaa-20ed2e1b9877 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3595cfbefb134e27a69dbefa059fc1b1'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:17.074973 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a6aed139-c793-4acf-baaa-20ed2e1b9877 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3595cfbefb134e27a69dbefa059fc1b1'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:17.081407 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 804/1609] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:06:16 2022] DELETE /identity/v3/users/3595cfbefb134e27a69dbefa059fc1b1 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:17.085988 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d74b5-54b7-4f48-957b-60f9f775bd41 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:17.086559 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d74b5-54b7-4f48-957b-60f9f775bd41 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:17.086559 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355d74b5-54b7-4f48-957b-60f9f775bd41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:17.106390 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-355d74b5-54b7-4f48-957b-60f9f775bd41 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:17.107678 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-355d74b5-54b7-4f48-957b-60f9f775bd41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:17.185977 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 806/1610] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:17.197497 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aed139-c793-4acf-baaa-20ed2e1b9877 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:17.198926 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a6aed139-c793-4acf-baaa-20ed2e1b9877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:17.264664 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6aed139-c793-4acf-baaa-20ed2e1b9877 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:17.268778 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a5424-4183-48b9-8529-3e37eeca75e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:17.270985 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a5424-4183-48b9-8529-3e37eeca75e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:17.271227 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a5424-4183-48b9-8529-3e37eeca75e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:17.271520 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36a5424-4183-48b9-8529-3e37eeca75e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:17.299749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a5424-4183-48b9-8529-3e37eeca75e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:17.303155 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36a5424-4183-48b9-8529-3e37eeca75e4 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:17.334020 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 805/1611] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:06:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:06:17.879244 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d74b5-54b7-4f48-957b-60f9f775bd41 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:17.880902 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-355d74b5-54b7-4f48-957b-60f9f775bd41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:17.902681 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355d74b5-54b7-4f48-957b-60f9f775bd41 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:17.910931 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec298d-f397-44f9-bbeb-969ef09a68f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:17.911632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec298d-f397-44f9-bbeb-969ef09a68f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:17.911903 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec298d-f397-44f9-bbeb-969ef09a68f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:17.912182 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ec298d-f397-44f9-bbeb-969ef09a68f9 admin admin] PATH_INFO: `/v3/projects/35f8e87c854e4587884430909632cad4` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:17.914041 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ec298d-f397-44f9-bbeb-969ef09a68f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35f8e87c854e4587884430909632cad4)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:17.915995 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ec298d-f397-44f9-bbeb-969ef09a68f9 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:17.965391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-69ec298d-f397-44f9-bbeb-969ef09a68f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35f8e87c854e4587884430909632cad4', 'request_id': 'req-69ec298d-f397-44f9-bbeb-969ef09a68f9'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:17.974096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-69ec298d-f397-44f9-bbeb-969ef09a68f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35f8e87c854e4587884430909632cad4', 'request_id': 'req-69ec298d-f397-44f9-bbeb-969ef09a68f9'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:17.981095 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-69ec298d-f397-44f9-bbeb-969ef09a68f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35f8e87c854e4587884430909632cad4', 'request_id': 'req-69ec298d-f397-44f9-bbeb-969ef09a68f9'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:17.995395 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-69ec298d-f397-44f9-bbeb-969ef09a68f9 admin admin] The token cache is being invalidate because project 35f8e87c854e4587884430909632cad4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:06:17.995946 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-69ec298d-f397-44f9-bbeb-969ef09a68f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:17.996408 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-69ec298d-f397-44f9-bbeb-969ef09a68f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:17.997941 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 807/1612] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:06:17 2022] DELETE /identity/v3/projects/35f8e87c854e4587884430909632cad4 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:18.002095 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a5424-4183-48b9-8529-3e37eeca75e4 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:18.017647 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36a5424-4183-48b9-8529-3e37eeca75e4 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:18.076438 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988b1762-a270-4b8e-8257-30bcea29aae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:18.077060 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b1762-a270-4b8e-8257-30bcea29aae3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:18.077344 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b1762-a270-4b8e-8257-30bcea29aae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:18.077620 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988b1762-a270-4b8e-8257-30bcea29aae3 admin admin] PATH_INFO: `/v3/users/bf1bfc78bfff415d818a8ef9baf5a06d` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:18.078768 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b1762-a270-4b8e-8257-30bcea29aae3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf1bfc78bfff415d818a8ef9baf5a06d)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:18.080676 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988b1762-a270-4b8e-8257-30bcea29aae3 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:18.125751 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-988b1762-a270-4b8e-8257-30bcea29aae3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf1bfc78bfff415d818a8ef9baf5a06d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:18.137047 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:18.138211 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-988b1762-a270-4b8e-8257-30bcea29aae3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf1bfc78bfff415d818a8ef9baf5a06d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:18.145873 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-988b1762-a270-4b8e-8257-30bcea29aae3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf1bfc78bfff415d818a8ef9baf5a06d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:18.146543 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-988b1762-a270-4b8e-8257-30bcea29aae3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf1bfc78bfff415d818a8ef9baf5a06d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:18.153973 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-988b1762-a270-4b8e-8257-30bcea29aae3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf1bfc78bfff415d818a8ef9baf5a06d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:18.159863 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 806/1613] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:06:18 2022] DELETE /identity/v3/users/bf1bfc78bfff415d818a8ef9baf5a06d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:18.421585 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec298d-f397-44f9-bbeb-969ef09a68f9 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:18.423120 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-69ec298d-f397-44f9-bbeb-969ef09a68f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:18.447059 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ec298d-f397-44f9-bbeb-969ef09a68f9 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:18.504950 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5677ec-35dc-4e1f-8adc-8a7dbf8a98b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:18.505514 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5677ec-35dc-4e1f-8adc-8a7dbf8a98b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:18.505796 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5677ec-35dc-4e1f-8adc-8a7dbf8a98b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:18.506071 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5677ec-35dc-4e1f-8adc-8a7dbf8a98b2 admin admin] PATH_INFO: `/v3/projects/75b1c8bae8d24e92a04d48dbebd9fcf5` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:18.507257 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5677ec-35dc-4e1f-8adc-8a7dbf8a98b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75b1c8bae8d24e92a04d48dbebd9fcf5)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:18.509166 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5677ec-35dc-4e1f-8adc-8a7dbf8a98b2 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:18.516985 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:18.557461 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-3d5677ec-35dc-4e1f-8adc-8a7dbf8a98b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75b1c8bae8d24e92a04d48dbebd9fcf5', 'request_id': 'req-3d5677ec-35dc-4e1f-8adc-8a7dbf8a98b2'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:18.564805 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-3d5677ec-35dc-4e1f-8adc-8a7dbf8a98b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75b1c8bae8d24e92a04d48dbebd9fcf5', 'request_id': 'req-3d5677ec-35dc-4e1f-8adc-8a7dbf8a98b2'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:18.570826 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-3d5677ec-35dc-4e1f-8adc-8a7dbf8a98b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75b1c8bae8d24e92a04d48dbebd9fcf5', 'request_id': 'req-3d5677ec-35dc-4e1f-8adc-8a7dbf8a98b2'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:18.581744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-3d5677ec-35dc-4e1f-8adc-8a7dbf8a98b2 admin admin] The token cache is being invalidate because project 75b1c8bae8d24e92a04d48dbebd9fcf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:06:18.582033 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-3d5677ec-35dc-4e1f-8adc-8a7dbf8a98b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:18.582695 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-3d5677ec-35dc-4e1f-8adc-8a7dbf8a98b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:18.584186 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 808/1614] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:06:18 2022] DELETE /identity/v3/projects/75b1c8bae8d24e92a04d48dbebd9fcf5 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:18.590290 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304ec89-8347-481a-b455-76042d3fb8ac None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:18.590606 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304ec89-8347-481a-b455-76042d3fb8ac None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:18.590879 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304ec89-8347-481a-b455-76042d3fb8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:18.610392 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-6304ec89-8347-481a-b455-76042d3fb8ac None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:18.611580 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-6304ec89-8347-481a-b455-76042d3fb8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:18.679677 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 807/1615] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:18.687385 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d95782-8fe6-44d1-94cc-154448f3e172 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:18.687700 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d95782-8fe6-44d1-94cc-154448f3e172 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:18.687967 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d95782-8fe6-44d1-94cc-154448f3e172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:18.838874 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-f6d95782-8fe6-44d1-94cc-154448f3e172 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:18.840143 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-f6d95782-8fe6-44d1-94cc-154448f3e172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:18.848513 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 809/1616] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:18.852418 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6304ec89-8347-481a-b455-76042d3fb8ac None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:18.863782 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6304ec89-8347-481a-b455-76042d3fb8ac None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:18.868980 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc518d7-4a75-4ca6-b679-0a81dbcf118e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:18.869585 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc518d7-4a75-4ca6-b679-0a81dbcf118e admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:18.869866 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc518d7-4a75-4ca6-b679-0a81dbcf118e admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:18.870141 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc518d7-4a75-4ca6-b679-0a81dbcf118e admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:18.870837 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc518d7-4a75-4ca6-b679-0a81dbcf118e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:06:18.871141 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc518d7-4a75-4ca6-b679-0a81dbcf118e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:18.872768 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc518d7-4a75-4ca6-b679-0a81dbcf118e admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:18.884286 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 808/1617] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:06:18.888513 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d95782-8fe6-44d1-94cc-154448f3e172 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:18.897877 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d95782-8fe6-44d1-94cc-154448f3e172 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:18.902380 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d99fe9-9d0c-4821-aca8-b1be5258ffe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:18.902975 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d99fe9-9d0c-4821-aca8-b1be5258ffe1 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:18.903270 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d99fe9-9d0c-4821-aca8-b1be5258ffe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:18.903532 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d99fe9-9d0c-4821-aca8-b1be5258ffe1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:18.904154 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d99fe9-9d0c-4821-aca8-b1be5258ffe1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:18.906030 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d99fe9-9d0c-4821-aca8-b1be5258ffe1 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:18.936181 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 810/1618] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:06:18 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:06:18.937649 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc518d7-4a75-4ca6-b679-0a81dbcf118e admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:18.947805 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc518d7-4a75-4ca6-b679-0a81dbcf118e admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:19.007960 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac083c67-55e9-4068-a997-b53747de367d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:19.011764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac083c67-55e9-4068-a997-b53747de367d admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:19.011764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac083c67-55e9-4068-a997-b53747de367d admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:19.011764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac083c67-55e9-4068-a997-b53747de367d admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:19.011764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac083c67-55e9-4068-a997-b53747de367d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:19.012413 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac083c67-55e9-4068-a997-b53747de367d admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:19.040192 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 809/1619] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:06:18 2022] POST /identity/v3/users => generated 285 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:06:19.041335 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d99fe9-9d0c-4821-aca8-b1be5258ffe1 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:19.052041 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d99fe9-9d0c-4821-aca8-b1be5258ffe1 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:19.057917 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959b1c09-8e81-424a-a948-3581ad3d8e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:19.060638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959b1c09-8e81-424a-a948-3581ad3d8e16 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:19.061002 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959b1c09-8e81-424a-a948-3581ad3d8e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:19.061352 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959b1c09-8e81-424a-a948-3581ad3d8e16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:19.062089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959b1c09-8e81-424a-a948-3581ad3d8e16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:19.063932 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959b1c09-8e81-424a-a948-3581ad3d8e16 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:19.076374 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 811/1620] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:06:19.079340 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac083c67-55e9-4068-a997-b53747de367d admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:19.089160 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac083c67-55e9-4068-a997-b53747de367d admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:19.093732 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c219abd-93ec-4561-9643-09f268418160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:19.094357 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c219abd-93ec-4561-9643-09f268418160 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:19.094640 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c219abd-93ec-4561-9643-09f268418160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:19.094914 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c219abd-93ec-4561-9643-09f268418160 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:19.095557 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c219abd-93ec-4561-9643-09f268418160 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:19.097151 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c219abd-93ec-4561-9643-09f268418160 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:19.108638 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 810/1621] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:06:19.112712 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959b1c09-8e81-424a-a948-3581ad3d8e16 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:19.123484 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959b1c09-8e81-424a-a948-3581ad3d8e16 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:19.128837 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908ccab-a2a4-4d76-be9c-c260fb793b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:19.129509 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908ccab-a2a4-4d76-be9c-c260fb793b16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:19.129790 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908ccab-a2a4-4d76-be9c-c260fb793b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:19.130682 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d908ccab-a2a4-4d76-be9c-c260fb793b16 admin admin] PATH_INFO: `/v3/projects/6d3ad69124de44b28429c979daba1585/users/bcfe8b111d264bf3998d143a705bf8bf/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:19.145713 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d908ccab-a2a4-4d76-be9c-c260fb793b16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d3ad69124de44b28429c979daba1585, user_id=bcfe8b111d264bf3998d143a705bf8bf, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:19.148474 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d908ccab-a2a4-4d76-be9c-c260fb793b16 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:19.164443 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 812/1622] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:06:19 2022] PUT /identity/v3/projects/6d3ad69124de44b28429c979daba1585/users/bcfe8b111d264bf3998d143a705bf8bf/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:06:19.176026 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45a1a4-ad49-4e5d-b463-44603b9c76ff None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:19.176315 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45a1a4-ad49-4e5d-b463-44603b9c76ff None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:19.176592 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45a1a4-ad49-4e5d-b463-44603b9c76ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:19.205741 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-cc45a1a4-ad49-4e5d-b463-44603b9c76ff None None] MFA Rules not processed for user `bcfe8b111d264bf3998d143a705bf8bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:19.207166 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-cc45a1a4-ad49-4e5d-b463-44603b9c76ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:19.250753 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:19.282599 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 811/1623] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:19.286936 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908ccab-a2a4-4d76-be9c-c260fb793b16 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:19.349469 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d908ccab-a2a4-4d76-be9c-c260fb793b16 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:19.354106 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f6c0a-944c-43e2-907b-6f1277dee964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:19.354740 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f6c0a-944c-43e2-907b-6f1277dee964 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:19.355017 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f6c0a-944c-43e2-907b-6f1277dee964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:19.355287 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85f6c0a-944c-43e2-907b-6f1277dee964 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:19.355915 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85f6c0a-944c-43e2-907b-6f1277dee964 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:19.357805 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85f6c0a-944c-43e2-907b-6f1277dee964 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:19.384847 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 813/1624] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:06:19 2022] POST /identity/v3/projects => generated 367 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:06:19.389024 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45a1a4-ad49-4e5d-b463-44603b9c76ff None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:19.399726 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45a1a4-ad49-4e5d-b463-44603b9c76ff None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:19.454790 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755444b8-d1c3-4022-a18d-80ac25d1e1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:19.455401 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755444b8-d1c3-4022-a18d-80ac25d1e1b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:19.455682 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755444b8-d1c3-4022-a18d-80ac25d1e1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:19.455953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755444b8-d1c3-4022-a18d-80ac25d1e1b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:19.456596 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755444b8-d1c3-4022-a18d-80ac25d1e1b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:19.458519 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755444b8-d1c3-4022-a18d-80ac25d1e1b8 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:19.482268 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 812/1625] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:06:19 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:06:19.486355 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f6c0a-944c-43e2-907b-6f1277dee964 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:19.496511 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85f6c0a-944c-43e2-907b-6f1277dee964 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:19.501044 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c2618-8584-4569-8173-c7ccfd4a7a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:19.501635 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c2618-8584-4569-8173-c7ccfd4a7a01 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:19.501915 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c2618-8584-4569-8173-c7ccfd4a7a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:19.502193 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c2618-8584-4569-8173-c7ccfd4a7a01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:19.502847 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c2618-8584-4569-8173-c7ccfd4a7a01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:19.504447 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c2618-8584-4569-8173-c7ccfd4a7a01 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:19.514067 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 814/1626] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:06:19.518054 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755444b8-d1c3-4022-a18d-80ac25d1e1b8 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:19.528207 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755444b8-d1c3-4022-a18d-80ac25d1e1b8 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:19.532790 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9422ebb3-e228-4bfe-bdf7-cf1067656f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:19.533373 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422ebb3-e228-4bfe-bdf7-cf1067656f84 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:19.533649 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422ebb3-e228-4bfe-bdf7-cf1067656f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:19.533920 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9422ebb3-e228-4bfe-bdf7-cf1067656f84 admin admin] PATH_INFO: `/v3/projects/41f5d64c233e48b7ada85e7e1b749cfc/users/e898be40dfa549229814f5f54ea7b4fe/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:19.548997 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9422ebb3-e228-4bfe-bdf7-cf1067656f84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41f5d64c233e48b7ada85e7e1b749cfc, user_id=e898be40dfa549229814f5f54ea7b4fe, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:19.551785 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9422ebb3-e228-4bfe-bdf7-cf1067656f84 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:19.564309 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 813/1627] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:06:19 2022] PUT /identity/v3/projects/41f5d64c233e48b7ada85e7e1b749cfc/users/e898be40dfa549229814f5f54ea7b4fe/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:19.568123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c2618-8584-4569-8173-c7ccfd4a7a01 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:19.578375 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c2618-8584-4569-8173-c7ccfd4a7a01 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:19.581921 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:19.636159 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb43aa98-0c60-4ff4-8814-5ea8e913e575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:19.636773 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43aa98-0c60-4ff4-8814-5ea8e913e575 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:19.637050 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43aa98-0c60-4ff4-8814-5ea8e913e575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:19.637318 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb43aa98-0c60-4ff4-8814-5ea8e913e575 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:19.637957 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb43aa98-0c60-4ff4-8814-5ea8e913e575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:19.639579 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb43aa98-0c60-4ff4-8814-5ea8e913e575 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:19.649833 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 815/1628] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:06:19.653923 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9422ebb3-e228-4bfe-bdf7-cf1067656f84 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:19.664998 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9422ebb3-e228-4bfe-bdf7-cf1067656f84 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:19.669632 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d6d7a4-7e15-4875-b99c-d7777416b54f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:19.670314 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6d7a4-7e15-4875-b99c-d7777416b54f admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:19.670595 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6d7a4-7e15-4875-b99c-d7777416b54f admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:19.670868 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d6d7a4-7e15-4875-b99c-d7777416b54f admin admin] PATH_INFO: `/v3/projects/41f5d64c233e48b7ada85e7e1b749cfc/users/e898be40dfa549229814f5f54ea7b4fe/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:19.672730 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d6d7a4-7e15-4875-b99c-d7777416b54f admin admin] RBAC: Authorizing `identity:create_grant(project_id=41f5d64c233e48b7ada85e7e1b749cfc, user_id=e898be40dfa549229814f5f54ea7b4fe, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:19.675395 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d6d7a4-7e15-4875-b99c-d7777416b54f admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:19.685434 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 814/1629] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:06:19 2022] PUT /identity/v3/projects/41f5d64c233e48b7ada85e7e1b749cfc/users/e898be40dfa549229814f5f54ea7b4fe/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:19.695594 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934518d5-2148-4bdb-90ca-be445d45e975 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:19.695930 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934518d5-2148-4bdb-90ca-be445d45e975 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:19.696227 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934518d5-2148-4bdb-90ca-be445d45e975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:19.724074 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-934518d5-2148-4bdb-90ca-be445d45e975 None None] MFA Rules not processed for user `e898be40dfa549229814f5f54ea7b4fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:19.726658 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-934518d5-2148-4bdb-90ca-be445d45e975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:19.799701 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 816/1630] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:19 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:19.812510 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d6d7a4-7e15-4875-b99c-d7777416b54f admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:19.814586 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e2d6d7a4-7e15-4875-b99c-d7777416b54f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:19.879587 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d6d7a4-7e15-4875-b99c-d7777416b54f admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:19.885700 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803223b9-c92a-4087-a491-db8772a708f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:19.886693 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803223b9-c92a-4087-a491-db8772a708f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:19.887533 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803223b9-c92a-4087-a491-db8772a708f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:19.887825 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803223b9-c92a-4087-a491-db8772a708f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:19.900851 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803223b9-c92a-4087-a491-db8772a708f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:19.902360 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803223b9-c92a-4087-a491-db8772a708f2 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:19.918087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 815/1631] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:06:19 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 20:06:19.971974 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934518d5-2148-4bdb-90ca-be445d45e975 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:19.973713 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-934518d5-2148-4bdb-90ca-be445d45e975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:19.989927 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934518d5-2148-4bdb-90ca-be445d45e975 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:19.993641 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2320202-95fd-4c41-8fe3-28704a6eeaa3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:19.994284 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2320202-95fd-4c41-8fe3-28704a6eeaa3 service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:19.994565 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2320202-95fd-4c41-8fe3-28704a6eeaa3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:19.994834 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2320202-95fd-4c41-8fe3-28704a6eeaa3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:20.089558 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2320202-95fd-4c41-8fe3-28704a6eeaa3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:20.091017 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2320202-95fd-4c41-8fe3-28704a6eeaa3 service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:20.100111 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 817/1632] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:06:19 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:06:23.464370 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803223b9-c92a-4087-a491-db8772a708f2 service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:23.496534 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:23.526543 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803223b9-c92a-4087-a491-db8772a708f2 service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:23.530958 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a9fe38-ca23-487f-a112-bf4801875baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:23.531564 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9fe38-ca23-487f-a112-bf4801875baa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:23.531837 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9fe38-ca23-487f-a112-bf4801875baa admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:23.532108 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a9fe38-ca23-487f-a112-bf4801875baa admin admin] PATH_INFO: `/v3/users/bcfe8b111d264bf3998d143a705bf8bf` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:23.533210 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a9fe38-ca23-487f-a112-bf4801875baa admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcfe8b111d264bf3998d143a705bf8bf)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:23.535149 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a9fe38-ca23-487f-a112-bf4801875baa admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:23.580279 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-14a9fe38-ca23-487f-a112-bf4801875baa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcfe8b111d264bf3998d143a705bf8bf'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:23.591885 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-14a9fe38-ca23-487f-a112-bf4801875baa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcfe8b111d264bf3998d143a705bf8bf'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:23.598804 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-14a9fe38-ca23-487f-a112-bf4801875baa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcfe8b111d264bf3998d143a705bf8bf'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:23.599426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-14a9fe38-ca23-487f-a112-bf4801875baa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcfe8b111d264bf3998d143a705bf8bf'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:23.606779 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-14a9fe38-ca23-487f-a112-bf4801875baa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcfe8b111d264bf3998d143a705bf8bf'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:23.614275 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 816/1633] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:06:23 2022] DELETE /identity/v3/users/bcfe8b111d264bf3998d143a705bf8bf => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:23.619254 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6fdbe5-5205-4c0a-bf9c-f282355578f9 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:23.619676 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6fdbe5-5205-4c0a-bf9c-f282355578f9 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:23.619676 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6fdbe5-5205-4c0a-bf9c-f282355578f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:23.640223 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-4b6fdbe5-5205-4c0a-bf9c-f282355578f9 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:23.641575 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-4b6fdbe5-5205-4c0a-bf9c-f282355578f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:23.686658 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:23.738169 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 818/1634] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:23.747890 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a9fe38-ca23-487f-a112-bf4801875baa admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:23.749305 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-14a9fe38-ca23-487f-a112-bf4801875baa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:23.809396 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a9fe38-ca23-487f-a112-bf4801875baa admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:23.813062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd9bbe7-cbc1-4425-83fd-12280c9a8e77 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:23.813685 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd9bbe7-cbc1-4425-83fd-12280c9a8e77 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:23.813968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd9bbe7-cbc1-4425-83fd-12280c9a8e77 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:23.814259 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd9bbe7-cbc1-4425-83fd-12280c9a8e77 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:23.826300 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd9bbe7-cbc1-4425-83fd-12280c9a8e77 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:23.828292 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd9bbe7-cbc1-4425-83fd-12280c9a8e77 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:23.844159 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 817/1635] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:06:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:06:24.228403 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6fdbe5-5205-4c0a-bf9c-f282355578f9 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:24.230183 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-4b6fdbe5-5205-4c0a-bf9c-f282355578f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:24.242498 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6fdbe5-5205-4c0a-bf9c-f282355578f9 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:24.247287 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e3afc-ef7d-4529-baaa-39c2fbc2e783 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:24.247960 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e3afc-ef7d-4529-baaa-39c2fbc2e783 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:24.248306 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e3afc-ef7d-4529-baaa-39c2fbc2e783 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:24.248518 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797e3afc-ef7d-4529-baaa-39c2fbc2e783 admin admin] PATH_INFO: `/v3/projects/6d3ad69124de44b28429c979daba1585` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:24.249991 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797e3afc-ef7d-4529-baaa-39c2fbc2e783 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d3ad69124de44b28429c979daba1585)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:24.258054 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797e3afc-ef7d-4529-baaa-39c2fbc2e783 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:24.330085 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-797e3afc-ef7d-4529-baaa-39c2fbc2e783 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d3ad69124de44b28429c979daba1585', 'request_id': 'req-797e3afc-ef7d-4529-baaa-39c2fbc2e783'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:24.340017 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-797e3afc-ef7d-4529-baaa-39c2fbc2e783 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d3ad69124de44b28429c979daba1585', 'request_id': 'req-797e3afc-ef7d-4529-baaa-39c2fbc2e783'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:24.348265 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-797e3afc-ef7d-4529-baaa-39c2fbc2e783 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d3ad69124de44b28429c979daba1585', 'request_id': 'req-797e3afc-ef7d-4529-baaa-39c2fbc2e783'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:24.362027 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-797e3afc-ef7d-4529-baaa-39c2fbc2e783 admin admin] The token cache is being invalidate because project 6d3ad69124de44b28429c979daba1585 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:06:24.362357 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-797e3afc-ef7d-4529-baaa-39c2fbc2e783 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:24.363017 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-797e3afc-ef7d-4529-baaa-39c2fbc2e783 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:24.364508 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 819/1636] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:06:24 2022] DELETE /identity/v3/projects/6d3ad69124de44b28429c979daba1585 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:24.368534 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd9bbe7-cbc1-4425-83fd-12280c9a8e77 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:24.378710 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd9bbe7-cbc1-4425-83fd-12280c9a8e77 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:24.427106 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:24.437033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf98647-5d7f-47c5-9823-417bd8577888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:24.437660 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf98647-5d7f-47c5-9823-417bd8577888 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:24.437938 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf98647-5d7f-47c5-9823-417bd8577888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:24.438209 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf98647-5d7f-47c5-9823-417bd8577888 admin admin] PATH_INFO: `/v3/users/e898be40dfa549229814f5f54ea7b4fe` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:24.439344 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf98647-5d7f-47c5-9823-417bd8577888 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e898be40dfa549229814f5f54ea7b4fe)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:24.441373 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf98647-5d7f-47c5-9823-417bd8577888 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:24.487006 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-cdf98647-5d7f-47c5-9823-417bd8577888 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e898be40dfa549229814f5f54ea7b4fe'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:24.499905 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-cdf98647-5d7f-47c5-9823-417bd8577888 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e898be40dfa549229814f5f54ea7b4fe'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:24.507721 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-cdf98647-5d7f-47c5-9823-417bd8577888 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e898be40dfa549229814f5f54ea7b4fe'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:24.508417 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-cdf98647-5d7f-47c5-9823-417bd8577888 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e898be40dfa549229814f5f54ea7b4fe'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:24.515410 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-cdf98647-5d7f-47c5-9823-417bd8577888 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e898be40dfa549229814f5f54ea7b4fe'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:24.521250 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 818/1637] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:06:24 2022] DELETE /identity/v3/users/e898be40dfa549229814f5f54ea7b4fe => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:24.794153 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e3afc-ef7d-4529-baaa-39c2fbc2e783 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:24.795689 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-797e3afc-ef7d-4529-baaa-39c2fbc2e783 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:24.809572 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797e3afc-ef7d-4529-baaa-39c2fbc2e783 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:24.867966 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6485bd2-9724-419b-a162-c65900442bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:24.868632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6485bd2-9724-419b-a162-c65900442bb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:24.868942 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6485bd2-9724-419b-a162-c65900442bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:24.869219 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6485bd2-9724-419b-a162-c65900442bb0 admin admin] PATH_INFO: `/v3/projects/41f5d64c233e48b7ada85e7e1b749cfc` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:24.870392 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6485bd2-9724-419b-a162-c65900442bb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41f5d64c233e48b7ada85e7e1b749cfc)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:24.872304 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6485bd2-9724-419b-a162-c65900442bb0 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:24.923545 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-d6485bd2-9724-419b-a162-c65900442bb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41f5d64c233e48b7ada85e7e1b749cfc', 'request_id': 'req-d6485bd2-9724-419b-a162-c65900442bb0'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:24.926585 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:24.931248 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-d6485bd2-9724-419b-a162-c65900442bb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41f5d64c233e48b7ada85e7e1b749cfc', 'request_id': 'req-d6485bd2-9724-419b-a162-c65900442bb0'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:24.939161 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-d6485bd2-9724-419b-a162-c65900442bb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41f5d64c233e48b7ada85e7e1b749cfc', 'request_id': 'req-d6485bd2-9724-419b-a162-c65900442bb0'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:24.954480 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-d6485bd2-9724-419b-a162-c65900442bb0 admin admin] The token cache is being invalidate because project 41f5d64c233e48b7ada85e7e1b749cfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:06:24.954801 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-d6485bd2-9724-419b-a162-c65900442bb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:24.956118 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-d6485bd2-9724-419b-a162-c65900442bb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:24.956963 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 820/1638] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:06:24 2022] DELETE /identity/v3/projects/41f5d64c233e48b7ada85e7e1b749cfc => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:24.963073 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe58c1-8a9e-4a21-99fd-bf65dc170179 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:24.963375 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe58c1-8a9e-4a21-99fd-bf65dc170179 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:24.963846 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffe58c1-8a9e-4a21-99fd-bf65dc170179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:24.985599 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-9ffe58c1-8a9e-4a21-99fd-bf65dc170179 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:24.986855 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9ffe58c1-8a9e-4a21-99fd-bf65dc170179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:25.058052 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 819/1639] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:25.065710 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99291bd3-0208-4761-84b9-e72ba3d10b54 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:25.065989 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99291bd3-0208-4761-84b9-e72ba3d10b54 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:25.066281 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99291bd3-0208-4761-84b9-e72ba3d10b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:25.092263 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-99291bd3-0208-4761-84b9-e72ba3d10b54 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:25.093490 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-99291bd3-0208-4761-84b9-e72ba3d10b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:25.102057 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 821/1640] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:25.106186 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe58c1-8a9e-4a21-99fd-bf65dc170179 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:25.116572 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffe58c1-8a9e-4a21-99fd-bf65dc170179 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:25.121499 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429a7e2-6621-4316-b97c-db7c4fec49c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:25.122221 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429a7e2-6621-4316-b97c-db7c4fec49c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:25.122409 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429a7e2-6621-4316-b97c-db7c4fec49c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:25.122788 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6429a7e2-6621-4316-b97c-db7c4fec49c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:25.123530 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6429a7e2-6621-4316-b97c-db7c4fec49c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:06:25.123987 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6429a7e2-6621-4316-b97c-db7c4fec49c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:25.125927 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6429a7e2-6621-4316-b97c-db7c4fec49c7 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:25.137960 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 820/1641] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:06:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 20:06:25.142307 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99291bd3-0208-4761-84b9-e72ba3d10b54 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:25.151649 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99291bd3-0208-4761-84b9-e72ba3d10b54 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:25.156250 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc29536-0963-4c13-abf3-9b9d243d8b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:25.156853 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc29536-0963-4c13-abf3-9b9d243d8b55 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:25.157132 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc29536-0963-4c13-abf3-9b9d243d8b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:25.157405 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc29536-0963-4c13-abf3-9b9d243d8b55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:25.158045 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc29536-0963-4c13-abf3-9b9d243d8b55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:25.159971 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc29536-0963-4c13-abf3-9b9d243d8b55 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:25.188074 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 822/1642] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:06:25 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:06:25.191920 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429a7e2-6621-4316-b97c-db7c4fec49c7 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:25.202339 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6429a7e2-6621-4316-b97c-db7c4fec49c7 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:25.257848 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd5121-ba5a-4aef-ba74-7caab6a4a8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:25.258485 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd5121-ba5a-4aef-ba74-7caab6a4a8ca admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:25.258769 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd5121-ba5a-4aef-ba74-7caab6a4a8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:25.259043 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fd5121-ba5a-4aef-ba74-7caab6a4a8ca admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:25.259680 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd5121-ba5a-4aef-ba74-7caab6a4a8ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:25.261601 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fd5121-ba5a-4aef-ba74-7caab6a4a8ca admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:25.269381 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:25.283043 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 821/1643] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:06:25 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:06:25.287137 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc29536-0963-4c13-abf3-9b9d243d8b55 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:25.297888 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc29536-0963-4c13-abf3-9b9d243d8b55 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:25.302499 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55a3dc-509f-42ca-ac69-66a938905c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:25.303167 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55a3dc-509f-42ca-ac69-66a938905c67 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:25.303442 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55a3dc-509f-42ca-ac69-66a938905c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:25.303706 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb55a3dc-509f-42ca-ac69-66a938905c67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:25.304333 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb55a3dc-509f-42ca-ac69-66a938905c67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:25.305911 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb55a3dc-509f-42ca-ac69-66a938905c67 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:25.320215 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 823/1644] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:06:25.323324 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd5121-ba5a-4aef-ba74-7caab6a4a8ca admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:25.333772 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fd5121-ba5a-4aef-ba74-7caab6a4a8ca admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:25.339691 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41c9dc3-45ec-4d98-92d2-1b55e142dc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:25.340311 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c9dc3-45ec-4d98-92d2-1b55e142dc4e admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:25.340597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c9dc3-45ec-4d98-92d2-1b55e142dc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:25.340869 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41c9dc3-45ec-4d98-92d2-1b55e142dc4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:25.341525 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c9dc3-45ec-4d98-92d2-1b55e142dc4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:25.343141 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41c9dc3-45ec-4d98-92d2-1b55e142dc4e admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:25.354327 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 822/1645] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:06:25.363082 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55a3dc-509f-42ca-ac69-66a938905c67 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:25.374536 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb55a3dc-509f-42ca-ac69-66a938905c67 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:25.381242 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf73090b-628f-46a4-a6ef-8af20fbe3502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:25.381907 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73090b-628f-46a4-a6ef-8af20fbe3502 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:25.382183 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73090b-628f-46a4-a6ef-8af20fbe3502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:25.382617 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73090b-628f-46a4-a6ef-8af20fbe3502 admin admin] PATH_INFO: `/v3/projects/6683478dd70140d193557aa8b3facf32/users/9fb3ac26243c4d438186e7a4b1baba5d/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:25.401602 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf73090b-628f-46a4-a6ef-8af20fbe3502 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6683478dd70140d193557aa8b3facf32, user_id=9fb3ac26243c4d438186e7a4b1baba5d, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:25.404297 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf73090b-628f-46a4-a6ef-8af20fbe3502 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:25.420537 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 824/1646] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:06:25 2022] PUT /identity/v3/projects/6683478dd70140d193557aa8b3facf32/users/9fb3ac26243c4d438186e7a4b1baba5d/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:25.432203 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2d096-029f-4291-8beb-7f936ceb2bd1 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:25.432497 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2d096-029f-4291-8beb-7f936ceb2bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:25.432770 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d2d096-029f-4291-8beb-7f936ceb2bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:25.463570 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-10d2d096-029f-4291-8beb-7f936ceb2bd1 None None] MFA Rules not processed for user `9fb3ac26243c4d438186e7a4b1baba5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:25.464812 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-10d2d096-029f-4291-8beb-7f936ceb2bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:25.538533 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 823/1647] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:25.543055 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf73090b-628f-46a4-a6ef-8af20fbe3502 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:25.606800 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf73090b-628f-46a4-a6ef-8af20fbe3502 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:25.611296 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc59d0-d8e5-49c7-9697-fc91bf64d3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:25.611896 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc59d0-d8e5-49c7-9697-fc91bf64d3dc admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:25.612173 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc59d0-d8e5-49c7-9697-fc91bf64d3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:25.612442 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc59d0-d8e5-49c7-9697-fc91bf64d3dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:25.613062 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdc59d0-d8e5-49c7-9697-fc91bf64d3dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:25.614985 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdc59d0-d8e5-49c7-9697-fc91bf64d3dc admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:25.641016 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 825/1648] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:06:25 2022] POST /identity/v3/projects => generated 381 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:06:25.645250 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2d096-029f-4291-8beb-7f936ceb2bd1 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:25.656307 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d2d096-029f-4291-8beb-7f936ceb2bd1 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:25.713857 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e220b-78da-4c8c-83e3-7135e6aa20f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:25.714500 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e220b-78da-4c8c-83e3-7135e6aa20f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:25.714775 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e220b-78da-4c8c-83e3-7135e6aa20f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:25.715045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232e220b-78da-4c8c-83e3-7135e6aa20f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:25.715682 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232e220b-78da-4c8c-83e3-7135e6aa20f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:25.717704 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232e220b-78da-4c8c-83e3-7135e6aa20f2 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:25.740718 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 824/1649] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:06:25 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:06:25.744641 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc59d0-d8e5-49c7-9697-fc91bf64d3dc admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:25.754844 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc59d0-d8e5-49c7-9697-fc91bf64d3dc admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:25.759959 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba30a5-108d-4d99-9ba3-29c83df590a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:25.760564 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba30a5-108d-4d99-9ba3-29c83df590a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:25.760846 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba30a5-108d-4d99-9ba3-29c83df590a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:25.761123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ba30a5-108d-4d99-9ba3-29c83df590a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:25.761777 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ba30a5-108d-4d99-9ba3-29c83df590a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:25.763399 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ba30a5-108d-4d99-9ba3-29c83df590a8 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:25.774588 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 826/1650] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:06:25.778457 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e220b-78da-4c8c-83e3-7135e6aa20f2 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:25.788253 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232e220b-78da-4c8c-83e3-7135e6aa20f2 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:25.792754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895f1d0-b7a0-4704-8cc7-9e22d43ae4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:25.793393 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895f1d0-b7a0-4704-8cc7-9e22d43ae4a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:25.793669 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895f1d0-b7a0-4704-8cc7-9e22d43ae4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:25.793951 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b895f1d0-b7a0-4704-8cc7-9e22d43ae4a7 admin admin] PATH_INFO: `/v3/projects/80cfdade69d74a64852435195b9854cc/users/3d5a47359f7a4030999effc104ef6991/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:25.809214 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b895f1d0-b7a0-4704-8cc7-9e22d43ae4a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80cfdade69d74a64852435195b9854cc, user_id=3d5a47359f7a4030999effc104ef6991, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:25.811915 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b895f1d0-b7a0-4704-8cc7-9e22d43ae4a7 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:25.825030 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 825/1651] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:06:25 2022] PUT /identity/v3/projects/80cfdade69d74a64852435195b9854cc/users/3d5a47359f7a4030999effc104ef6991/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:06:25.828856 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba30a5-108d-4d99-9ba3-29c83df590a8 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:25.839543 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ba30a5-108d-4d99-9ba3-29c83df590a8 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:25.884483 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:25.895128 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb6d8c-43d9-4572-a628-a84569a9f322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:25.895728 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb6d8c-43d9-4572-a628-a84569a9f322 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:25.896034 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb6d8c-43d9-4572-a628-a84569a9f322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:25.896305 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb6d8c-43d9-4572-a628-a84569a9f322 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:25.897471 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb6d8c-43d9-4572-a628-a84569a9f322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:25.899094 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb6d8c-43d9-4572-a628-a84569a9f322 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:25.908546 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 827/1652] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:25 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:06:25.912897 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895f1d0-b7a0-4704-8cc7-9e22d43ae4a7 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:25.922965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b895f1d0-b7a0-4704-8cc7-9e22d43ae4a7 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:25.927405 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96730578-1768-4a84-98fb-16101f5de579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:25.928077 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96730578-1768-4a84-98fb-16101f5de579 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:25.928356 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96730578-1768-4a84-98fb-16101f5de579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:25.928623 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96730578-1768-4a84-98fb-16101f5de579 admin admin] PATH_INFO: `/v3/projects/80cfdade69d74a64852435195b9854cc/users/3d5a47359f7a4030999effc104ef6991/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:25.930558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96730578-1768-4a84-98fb-16101f5de579 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80cfdade69d74a64852435195b9854cc, user_id=3d5a47359f7a4030999effc104ef6991, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:25.933160 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96730578-1768-4a84-98fb-16101f5de579 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:25.943014 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 826/1653] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:06:25 2022] PUT /identity/v3/projects/80cfdade69d74a64852435195b9854cc/users/3d5a47359f7a4030999effc104ef6991/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:25.953798 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65ec15-6a5b-48d1-9616-36cf441ae042 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:25.954094 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65ec15-6a5b-48d1-9616-36cf441ae042 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:25.954416 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e65ec15-6a5b-48d1-9616-36cf441ae042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:25.981749 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-2e65ec15-6a5b-48d1-9616-36cf441ae042 None None] MFA Rules not processed for user `3d5a47359f7a4030999effc104ef6991`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:25.982988 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-2e65ec15-6a5b-48d1-9616-36cf441ae042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:26.051050 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 828/1654] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:25 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:26.063523 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96730578-1768-4a84-98fb-16101f5de579 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:26.065197 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-96730578-1768-4a84-98fb-16101f5de579 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:26.088285 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:26.129629 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96730578-1768-4a84-98fb-16101f5de579 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:26.134281 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4d613-c24c-4e88-83fe-4b71460b441d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:26.135220 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4d613-c24c-4e88-83fe-4b71460b441d service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:26.135220 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4d613-c24c-4e88-83fe-4b71460b441d service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:26.135737 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd4d613-c24c-4e88-83fe-4b71460b441d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:26.146287 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd4d613-c24c-4e88-83fe-4b71460b441d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:26.147088 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd4d613-c24c-4e88-83fe-4b71460b441d service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:26.156663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 827/1655] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:06:26 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:06:26.477410 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e65ec15-6a5b-48d1-9616-36cf441ae042 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:26.486352 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e65ec15-6a5b-48d1-9616-36cf441ae042 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:26.491014 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6786f0-16b2-4f8b-9262-cfabc1bdeda8 tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d5a47359f7a4030999effc104ef6991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80cfdade69d74a64852435195b9854cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:26.491615 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6786f0-16b2-4f8b-9262-cfabc1bdeda8 tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:26.491899 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6786f0-16b2-4f8b-9262-cfabc1bdeda8 tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:26.492175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6786f0-16b2-4f8b-9262-cfabc1bdeda8 tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:26.492872 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6786f0-16b2-4f8b-9262-cfabc1bdeda8 tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:06:26.493109 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6786f0-16b2-4f8b-9262-cfabc1bdeda8 tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:26.494716 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6786f0-16b2-4f8b-9262-cfabc1bdeda8 tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:26.510135 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 829/1656] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:06:26.512466 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4d613-c24c-4e88-83fe-4b71460b441d service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:26.522462 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd4d613-c24c-4e88-83fe-4b71460b441d service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:26.527212 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d5f79-26f5-4d50-846e-a30d1b98449e tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d5a47359f7a4030999effc104ef6991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80cfdade69d74a64852435195b9854cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:26.527928 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d5f79-26f5-4d50-846e-a30d1b98449e tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:26.528368 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d5f79-26f5-4d50-846e-a30d1b98449e tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:26.528645 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819d5f79-26f5-4d50-846e-a30d1b98449e tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:26.529476 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819d5f79-26f5-4d50-846e-a30d1b98449e tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:26.531492 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819d5f79-26f5-4d50-846e-a30d1b98449e tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:26.562619 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 828/1657] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:06:26 2022] POST /identity/v3/projects => generated 354 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:06:26.563649 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6786f0-16b2-4f8b-9262-cfabc1bdeda8 tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:26.581570 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6786f0-16b2-4f8b-9262-cfabc1bdeda8 tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:26.646009 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e5a20-0d22-490d-b0c6-bf75fe0fd72e tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d5a47359f7a4030999effc104ef6991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80cfdade69d74a64852435195b9854cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:26.646720 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e5a20-0d22-490d-b0c6-bf75fe0fd72e tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:26.647018 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e5a20-0d22-490d-b0c6-bf75fe0fd72e tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:26.647275 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e5a20-0d22-490d-b0c6-bf75fe0fd72e tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:26.647923 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e5a20-0d22-490d-b0c6-bf75fe0fd72e tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:06:26.648223 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e5a20-0d22-490d-b0c6-bf75fe0fd72e tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:26.649792 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e5a20-0d22-490d-b0c6-bf75fe0fd72e tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:26.672409 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 830/1658] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 20:06:26.708152 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d5f79-26f5-4d50-846e-a30d1b98449e tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:26.719220 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819d5f79-26f5-4d50-846e-a30d1b98449e tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:26.723743 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54db4560-4bee-4752-b199-0c1b8d4e1171 tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3d5a47359f7a4030999effc104ef6991', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '80cfdade69d74a64852435195b9854cc', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:26.724374 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db4560-4bee-4752-b199-0c1b8d4e1171 tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:26.725168 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db4560-4bee-4752-b199-0c1b8d4e1171 tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:26.725490 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54db4560-4bee-4752-b199-0c1b8d4e1171 tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] PATH_INFO: `/v3/projects/3a47c08d49f7445d8285776d118770c1` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:26.726629 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54db4560-4bee-4752-b199-0c1b8d4e1171 tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3a47c08d49f7445d8285776d118770c1)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:26.728547 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54db4560-4bee-4752-b199-0c1b8d4e1171 tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:26.789866 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-54db4560-4bee-4752-b199-0c1b8d4e1171 tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a47c08d49f7445d8285776d118770c1', 'request_id': 'req-54db4560-4bee-4752-b199-0c1b8d4e1171'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:26.799909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-54db4560-4bee-4752-b199-0c1b8d4e1171 tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a47c08d49f7445d8285776d118770c1', 'request_id': 'req-54db4560-4bee-4752-b199-0c1b8d4e1171'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:26.806268 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-54db4560-4bee-4752-b199-0c1b8d4e1171 tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a47c08d49f7445d8285776d118770c1', 'request_id': 'req-54db4560-4bee-4752-b199-0c1b8d4e1171'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:26.819439 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-54db4560-4bee-4752-b199-0c1b8d4e1171 tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] The token cache is being invalidate because project 3a47c08d49f7445d8285776d118770c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:06:26.819731 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-54db4560-4bee-4752-b199-0c1b8d4e1171 tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:26.820527 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-54db4560-4bee-4752-b199-0c1b8d4e1171 tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:26.822081 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 829/1659] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:06:26 2022] DELETE /identity/v3/projects/3a47c08d49f7445d8285776d118770c1 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:27.010086 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e5a20-0d22-490d-b0c6-bf75fe0fd72e tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:27.011972 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-639e5a20-0d22-490d-b0c6-bf75fe0fd72e tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:27.064223 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:27.076790 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e5a20-0d22-490d-b0c6-bf75fe0fd72e tempest-VolumeQuotaClassesTest-1446504328 tempest-VolumeQuotaClassesTest-1446504328-project-admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:27.131047 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1dfa88-d4d0-4f37-bdc1-ee24584ba763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:27.131668 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1dfa88-d4d0-4f37-bdc1-ee24584ba763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:27.131944 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1dfa88-d4d0-4f37-bdc1-ee24584ba763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:27.132216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1dfa88-d4d0-4f37-bdc1-ee24584ba763 admin admin] PATH_INFO: `/v3/users/9fb3ac26243c4d438186e7a4b1baba5d` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:27.133318 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1dfa88-d4d0-4f37-bdc1-ee24584ba763 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9fb3ac26243c4d438186e7a4b1baba5d)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:27.135372 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1dfa88-d4d0-4f37-bdc1-ee24584ba763 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:27.181434 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ff1dfa88-d4d0-4f37-bdc1-ee24584ba763 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9fb3ac26243c4d438186e7a4b1baba5d'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:27.193015 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ff1dfa88-d4d0-4f37-bdc1-ee24584ba763 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9fb3ac26243c4d438186e7a4b1baba5d'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:27.200222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ff1dfa88-d4d0-4f37-bdc1-ee24584ba763 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9fb3ac26243c4d438186e7a4b1baba5d'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:27.207804 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ff1dfa88-d4d0-4f37-bdc1-ee24584ba763 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9fb3ac26243c4d438186e7a4b1baba5d'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:27.208475 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ff1dfa88-d4d0-4f37-bdc1-ee24584ba763 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9fb3ac26243c4d438186e7a4b1baba5d'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:27.215199 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 831/1660] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:06:27 2022] DELETE /identity/v3/users/9fb3ac26243c4d438186e7a4b1baba5d => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:27.219489 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eceb10-5668-4e7c-aac3-47ba0c52a553 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:27.219848 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eceb10-5668-4e7c-aac3-47ba0c52a553 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:27.220194 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eceb10-5668-4e7c-aac3-47ba0c52a553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:27.241000 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-b9eceb10-5668-4e7c-aac3-47ba0c52a553 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:27.242136 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-b9eceb10-5668-4e7c-aac3-47ba0c52a553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:27.298929 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 830/1661] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:27.308648 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1dfa88-d4d0-4f37-bdc1-ee24584ba763 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:27.310516 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-ff1dfa88-d4d0-4f37-bdc1-ee24584ba763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:27.383319 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1dfa88-d4d0-4f37-bdc1-ee24584ba763 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:27.386983 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b27c9a-993a-4458-af30-e8ef298e621e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:27.387589 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b27c9a-993a-4458-af30-e8ef298e621e service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:27.387866 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b27c9a-993a-4458-af30-e8ef298e621e service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:27.388136 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b27c9a-993a-4458-af30-e8ef298e621e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:27.398022 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b27c9a-993a-4458-af30-e8ef298e621e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:27.399482 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b27c9a-993a-4458-af30-e8ef298e621e service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:27.409659 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 832/1662] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:06:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:06:28.272132 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eceb10-5668-4e7c-aac3-47ba0c52a553 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:28.273688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-b9eceb10-5668-4e7c-aac3-47ba0c52a553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:28.285762 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eceb10-5668-4e7c-aac3-47ba0c52a553 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:28.290823 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c835d-d397-4271-bcdf-f94deee267d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:28.291460 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c835d-d397-4271-bcdf-f94deee267d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:28.291739 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c835d-d397-4271-bcdf-f94deee267d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:28.292015 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3c835d-d397-4271-bcdf-f94deee267d6 admin admin] PATH_INFO: `/v3/projects/6683478dd70140d193557aa8b3facf32` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:28.293237 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3c835d-d397-4271-bcdf-f94deee267d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6683478dd70140d193557aa8b3facf32)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:28.299410 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3c835d-d397-4271-bcdf-f94deee267d6 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:28.321810 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:28.349104 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-8e3c835d-d397-4271-bcdf-f94deee267d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6683478dd70140d193557aa8b3facf32', 'request_id': 'req-8e3c835d-d397-4271-bcdf-f94deee267d6'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:28.356796 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-8e3c835d-d397-4271-bcdf-f94deee267d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6683478dd70140d193557aa8b3facf32', 'request_id': 'req-8e3c835d-d397-4271-bcdf-f94deee267d6'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:28.362489 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-8e3c835d-d397-4271-bcdf-f94deee267d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6683478dd70140d193557aa8b3facf32', 'request_id': 'req-8e3c835d-d397-4271-bcdf-f94deee267d6'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:28.374235 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-8e3c835d-d397-4271-bcdf-f94deee267d6 admin admin] The token cache is being invalidate because project 6683478dd70140d193557aa8b3facf32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:06:28.374557 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-8e3c835d-d397-4271-bcdf-f94deee267d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:28.375182 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-8e3c835d-d397-4271-bcdf-f94deee267d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:28.376657 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 831/1663] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:06:28 2022] DELETE /identity/v3/projects/6683478dd70140d193557aa8b3facf32 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:28.380647 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b27c9a-993a-4458-af30-e8ef298e621e service neutron] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:28.390215 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b27c9a-993a-4458-af30-e8ef298e621e service neutron] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:28.409216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:28.449395 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3626317-8281-4c46-bb55-9229ec7a475b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:28.450060 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3626317-8281-4c46-bb55-9229ec7a475b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:28.450372 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3626317-8281-4c46-bb55-9229ec7a475b admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:28.450651 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3626317-8281-4c46-bb55-9229ec7a475b admin admin] PATH_INFO: `/v3/users/3d5a47359f7a4030999effc104ef6991` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:28.451767 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3626317-8281-4c46-bb55-9229ec7a475b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d5a47359f7a4030999effc104ef6991)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:28.453678 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3626317-8281-4c46-bb55-9229ec7a475b admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:28.498150 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-d3626317-8281-4c46-bb55-9229ec7a475b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d5a47359f7a4030999effc104ef6991'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:28.510903 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-d3626317-8281-4c46-bb55-9229ec7a475b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d5a47359f7a4030999effc104ef6991'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:28.517651 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-d3626317-8281-4c46-bb55-9229ec7a475b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d5a47359f7a4030999effc104ef6991'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:28.524955 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-d3626317-8281-4c46-bb55-9229ec7a475b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d5a47359f7a4030999effc104ef6991'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:28.525683 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-d3626317-8281-4c46-bb55-9229ec7a475b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d5a47359f7a4030999effc104ef6991'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:28.532737 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 833/1664] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:06:28 2022] DELETE /identity/v3/users/3d5a47359f7a4030999effc104ef6991 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:28.798905 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c835d-d397-4271-bcdf-f94deee267d6 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:28.800459 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-8e3c835d-d397-4271-bcdf-f94deee267d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:28.813944 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3c835d-d397-4271-bcdf-f94deee267d6 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:28.884271 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3152d-6ed2-482e-8ebb-fd8ebc67a187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:28.884919 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3152d-6ed2-482e-8ebb-fd8ebc67a187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:28.885378 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3152d-6ed2-482e-8ebb-fd8ebc67a187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:28.885667 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3152d-6ed2-482e-8ebb-fd8ebc67a187 admin admin] PATH_INFO: `/v3/projects/80cfdade69d74a64852435195b9854cc` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:28.886825 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b3152d-6ed2-482e-8ebb-fd8ebc67a187 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80cfdade69d74a64852435195b9854cc)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:28.888740 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b3152d-6ed2-482e-8ebb-fd8ebc67a187 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:28.943983 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-c2b3152d-6ed2-482e-8ebb-fd8ebc67a187 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80cfdade69d74a64852435195b9854cc', 'request_id': 'req-c2b3152d-6ed2-482e-8ebb-fd8ebc67a187'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:28.959912 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-c2b3152d-6ed2-482e-8ebb-fd8ebc67a187 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80cfdade69d74a64852435195b9854cc', 'request_id': 'req-c2b3152d-6ed2-482e-8ebb-fd8ebc67a187'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:28.974183 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-c2b3152d-6ed2-482e-8ebb-fd8ebc67a187 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80cfdade69d74a64852435195b9854cc', 'request_id': 'req-c2b3152d-6ed2-482e-8ebb-fd8ebc67a187'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:28.986464 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-c2b3152d-6ed2-482e-8ebb-fd8ebc67a187 admin admin] The token cache is being invalidate because project 80cfdade69d74a64852435195b9854cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:06:28.986762 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-c2b3152d-6ed2-482e-8ebb-fd8ebc67a187 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:28.987461 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-c2b3152d-6ed2-482e-8ebb-fd8ebc67a187 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:28.988963 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 832/1665] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:06:28 2022] DELETE /identity/v3/projects/80cfdade69d74a64852435195b9854cc => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:06:28.993848 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf39dbb-71f1-4d10-84c8-92225ecc71da None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:28.994029 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf39dbb-71f1-4d10-84c8-92225ecc71da None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:28.994334 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf39dbb-71f1-4d10-84c8-92225ecc71da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:29.014200 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-1bf39dbb-71f1-4d10-84c8-92225ecc71da None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:29.016062 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-1bf39dbb-71f1-4d10-84c8-92225ecc71da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:29.080614 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 834/1666] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:06:29.088578 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f679d-a095-4ea9-a1e7-78653bbe208a None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:29.088864 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f679d-a095-4ea9-a1e7-78653bbe208a None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:29.089143 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46f679d-a095-4ea9-a1e7-78653bbe208a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:29.108647 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-c46f679d-a095-4ea9-a1e7-78653bbe208a None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:29.109945 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c46f679d-a095-4ea9-a1e7-78653bbe208a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:29.118354 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 833/1667] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:06:29.122490 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf39dbb-71f1-4d10-84c8-92225ecc71da None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:29.132613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf39dbb-71f1-4d10-84c8-92225ecc71da None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:29.137120 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae87911-fd19-40ff-8694-da338226afde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:29.137714 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae87911-fd19-40ff-8694-da338226afde admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:29.137995 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae87911-fd19-40ff-8694-da338226afde admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:29.138301 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae87911-fd19-40ff-8694-da338226afde admin admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:29.138984 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae87911-fd19-40ff-8694-da338226afde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:06:29.139289 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae87911-fd19-40ff-8694-da338226afde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:29.140866 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae87911-fd19-40ff-8694-da338226afde admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:29.152244 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 835/1668] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:06:29.156292 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f679d-a095-4ea9-a1e7-78653bbe208a None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:29.165255 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46f679d-a095-4ea9-a1e7-78653bbe208a None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:29.169849 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05b13c-bd7d-4fd3-a4c1-402ed1c9a6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:29.170475 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05b13c-bd7d-4fd3-a4c1-402ed1c9a6b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:29.170756 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05b13c-bd7d-4fd3-a4c1-402ed1c9a6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:29.171029 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a05b13c-bd7d-4fd3-a4c1-402ed1c9a6b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:29.171666 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05b13c-bd7d-4fd3-a4c1-402ed1c9a6b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:29.173547 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a05b13c-bd7d-4fd3-a4c1-402ed1c9a6b7 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:29.201652 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 834/1669] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:06:29 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:06:29.205964 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae87911-fd19-40ff-8694-da338226afde admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:29.216625 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae87911-fd19-40ff-8694-da338226afde admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:29.245020 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:29.286046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d78d9d-4670-4a84-af20-0993ad8f8074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:29.286696 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d78d9d-4670-4a84-af20-0993ad8f8074 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:29.286975 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d78d9d-4670-4a84-af20-0993ad8f8074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:29.287286 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d78d9d-4670-4a84-af20-0993ad8f8074 admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:29.287969 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d78d9d-4670-4a84-af20-0993ad8f8074 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:29.290034 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d78d9d-4670-4a84-af20-0993ad8f8074 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:29.314134 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 836/1670] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:06:29 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:06:29.318042 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05b13c-bd7d-4fd3-a4c1-402ed1c9a6b7 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:29.329576 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a05b13c-bd7d-4fd3-a4c1-402ed1c9a6b7 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:29.336321 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc531d19-c821-49a2-8d43-57a3c4805a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:29.337492 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc531d19-c821-49a2-8d43-57a3c4805a51 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:29.337492 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc531d19-c821-49a2-8d43-57a3c4805a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:29.337492 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc531d19-c821-49a2-8d43-57a3c4805a51 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:29.338126 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc531d19-c821-49a2-8d43-57a3c4805a51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:29.339760 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc531d19-c821-49a2-8d43-57a3c4805a51 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:29.342376 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:29.354758 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 835/1671] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:06:29.358305 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d78d9d-4670-4a84-af20-0993ad8f8074 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:29.369001 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d78d9d-4670-4a84-af20-0993ad8f8074 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:29.373824 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27658289-8e80-4f45-9d12-09e05d0b9130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:29.374479 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27658289-8e80-4f45-9d12-09e05d0b9130 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:29.374763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27658289-8e80-4f45-9d12-09e05d0b9130 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:29.375036 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27658289-8e80-4f45-9d12-09e05d0b9130 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:29.375687 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27658289-8e80-4f45-9d12-09e05d0b9130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:29.377287 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27658289-8e80-4f45-9d12-09e05d0b9130 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:29.391544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 837/1672] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 20:06:29.393571 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc531d19-c821-49a2-8d43-57a3c4805a51 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:29.404380 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc531d19-c821-49a2-8d43-57a3c4805a51 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:29.408961 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade1c11-929b-4d90-8cad-7cfcb7dad6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:29.409600 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade1c11-929b-4d90-8cad-7cfcb7dad6f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:29.409951 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade1c11-929b-4d90-8cad-7cfcb7dad6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:29.410213 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eade1c11-929b-4d90-8cad-7cfcb7dad6f6 admin admin] PATH_INFO: `/v3/projects/47032be45c734a00973adcbfa5696c37/users/3b32d4fd7c8740229cacf6b5c90ac781/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:29.425716 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eade1c11-929b-4d90-8cad-7cfcb7dad6f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47032be45c734a00973adcbfa5696c37, user_id=3b32d4fd7c8740229cacf6b5c90ac781, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:29.429166 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eade1c11-929b-4d90-8cad-7cfcb7dad6f6 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:29.448716 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 836/1673] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:06:29 2022] PUT /identity/v3/projects/47032be45c734a00973adcbfa5696c37/users/3b32d4fd7c8740229cacf6b5c90ac781/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:29.465667 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75ed8d-8539-434d-86ea-7cddc744cd57 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:29.466028 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75ed8d-8539-434d-86ea-7cddc744cd57 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:29.466427 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e75ed8d-8539-434d-86ea-7cddc744cd57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:29.494574 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-2e75ed8d-8539-434d-86ea-7cddc744cd57 None None] MFA Rules not processed for user `3b32d4fd7c8740229cacf6b5c90ac781`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:29.495780 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-2e75ed8d-8539-434d-86ea-7cddc744cd57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:29.559111 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 838/1674] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:29.563721 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade1c11-929b-4d90-8cad-7cfcb7dad6f6 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:29.624779 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eade1c11-929b-4d90-8cad-7cfcb7dad6f6 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:29.629403 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ca60c5-3c64-43a8-a18e-94264d6564fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:29.630005 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca60c5-3c64-43a8-a18e-94264d6564fb admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:29.630313 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca60c5-3c64-43a8-a18e-94264d6564fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:29.630596 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca60c5-3c64-43a8-a18e-94264d6564fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:29.631265 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ca60c5-3c64-43a8-a18e-94264d6564fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:29.633196 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ca60c5-3c64-43a8-a18e-94264d6564fb admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:29.658949 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 837/1675] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:06:29 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:06:29.663411 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e75ed8d-8539-434d-86ea-7cddc744cd57 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:29.673976 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e75ed8d-8539-434d-86ea-7cddc744cd57 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:29.729285 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013fa4f2-e792-427e-9ca7-7812eedafcd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:29.729899 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013fa4f2-e792-427e-9ca7-7812eedafcd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:29.730183 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013fa4f2-e792-427e-9ca7-7812eedafcd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:29.730492 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013fa4f2-e792-427e-9ca7-7812eedafcd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:29.731149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013fa4f2-e792-427e-9ca7-7812eedafcd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:29.733069 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013fa4f2-e792-427e-9ca7-7812eedafcd0 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:29.758734 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 839/1676] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:06:29 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:06:29.759941 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ca60c5-3c64-43a8-a18e-94264d6564fb admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:29.771650 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ca60c5-3c64-43a8-a18e-94264d6564fb admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:29.776432 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d4fcc-d7e3-4c01-8400-103082895d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:29.777095 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d4fcc-d7e3-4c01-8400-103082895d90 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:29.777328 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d4fcc-d7e3-4c01-8400-103082895d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:29.777603 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d4fcc-d7e3-4c01-8400-103082895d90 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:29.778283 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721d4fcc-d7e3-4c01-8400-103082895d90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:29.779862 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721d4fcc-d7e3-4c01-8400-103082895d90 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:29.794180 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 838/1677] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 20:06:29.795509 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013fa4f2-e792-427e-9ca7-7812eedafcd0 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:29.806388 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013fa4f2-e792-427e-9ca7-7812eedafcd0 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:29.811263 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69714e0c-78d7-4883-9ba6-ac7fed015930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:29.811807 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69714e0c-78d7-4883-9ba6-ac7fed015930 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:29.812085 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69714e0c-78d7-4883-9ba6-ac7fed015930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:29.812374 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69714e0c-78d7-4883-9ba6-ac7fed015930 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:29.813019 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69714e0c-78d7-4883-9ba6-ac7fed015930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:29.814632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69714e0c-78d7-4883-9ba6-ac7fed015930 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:29.824774 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 840/1678] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:06:29.828936 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d4fcc-d7e3-4c01-8400-103082895d90 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:29.839204 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d4fcc-d7e3-4c01-8400-103082895d90 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:29.843724 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea64160-e66b-41ef-a624-99497bc1fd5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:29.844364 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea64160-e66b-41ef-a624-99497bc1fd5e admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:29.844643 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea64160-e66b-41ef-a624-99497bc1fd5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:29.844917 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea64160-e66b-41ef-a624-99497bc1fd5e admin admin] PATH_INFO: `/v3/projects/200d1d7a5f4b4f4a9ecdda09cf8010a7/users/59735e50eea34cc398abd28921df601e/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:29.859966 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea64160-e66b-41ef-a624-99497bc1fd5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=200d1d7a5f4b4f4a9ecdda09cf8010a7, user_id=59735e50eea34cc398abd28921df601e, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:29.862653 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea64160-e66b-41ef-a624-99497bc1fd5e admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:29.875569 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 839/1679] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:06:29 2022] PUT /identity/v3/projects/200d1d7a5f4b4f4a9ecdda09cf8010a7/users/59735e50eea34cc398abd28921df601e/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:29.885583 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24560a34-0fd7-4645-93ad-d57f62734d36 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:29.885891 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24560a34-0fd7-4645-93ad-d57f62734d36 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:29.886171 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24560a34-0fd7-4645-93ad-d57f62734d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:29.914606 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-24560a34-0fd7-4645-93ad-d57f62734d36 None None] MFA Rules not processed for user `59735e50eea34cc398abd28921df601e`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:29.915814 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-24560a34-0fd7-4645-93ad-d57f62734d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:29.919728 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:29.981495 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 841/1680] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:29.988943 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea64160-e66b-41ef-a624-99497bc1fd5e admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:30.054390 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea64160-e66b-41ef-a624-99497bc1fd5e admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:30.059106 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807a753-0acf-4e64-ba14-2dbac975f931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:30.059718 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807a753-0acf-4e64-ba14-2dbac975f931 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:30.060002 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807a753-0acf-4e64-ba14-2dbac975f931 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:30.060287 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8807a753-0acf-4e64-ba14-2dbac975f931 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:30.060980 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8807a753-0acf-4e64-ba14-2dbac975f931 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:30.063174 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8807a753-0acf-4e64-ba14-2dbac975f931 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:30.092132 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 840/1681] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:06:29 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:06:30.096722 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24560a34-0fd7-4645-93ad-d57f62734d36 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:30.108116 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24560a34-0fd7-4645-93ad-d57f62734d36 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:30.167075 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ac96f-9b8b-4e75-b138-2502103ef717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:30.167697 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ac96f-9b8b-4e75-b138-2502103ef717 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:30.167979 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ac96f-9b8b-4e75-b138-2502103ef717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:30.168254 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7ac96f-9b8b-4e75-b138-2502103ef717 admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:30.168901 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7ac96f-9b8b-4e75-b138-2502103ef717 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:30.170987 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7ac96f-9b8b-4e75-b138-2502103ef717 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:30.198765 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 842/1682] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:06:30 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:06:30.200488 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807a753-0acf-4e64-ba14-2dbac975f931 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:30.212485 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8807a753-0acf-4e64-ba14-2dbac975f931 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:30.217427 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338cca45-074a-436f-9324-3799ef6bca6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:30.218037 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338cca45-074a-436f-9324-3799ef6bca6c admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:30.218359 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338cca45-074a-436f-9324-3799ef6bca6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:30.218633 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338cca45-074a-436f-9324-3799ef6bca6c admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:30.219310 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338cca45-074a-436f-9324-3799ef6bca6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:30.220961 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338cca45-074a-436f-9324-3799ef6bca6c admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:30.223635 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:30.233723 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 841/1683] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:06:30.238034 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ac96f-9b8b-4e75-b138-2502103ef717 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:30.249035 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7ac96f-9b8b-4e75-b138-2502103ef717 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:30.253668 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589360c5-21fa-4409-865e-f2313701116f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:30.254345 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589360c5-21fa-4409-865e-f2313701116f admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:30.254623 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589360c5-21fa-4409-865e-f2313701116f admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:30.254894 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589360c5-21fa-4409-865e-f2313701116f admin admin] PATH_INFO: `/v3/projects/b96e70dba40f475891ea2a40158cbf3e/users/666692a600484d7db9f8e4b3b8427bab/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:30.270627 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589360c5-21fa-4409-865e-f2313701116f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b96e70dba40f475891ea2a40158cbf3e, user_id=666692a600484d7db9f8e4b3b8427bab, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:30.273409 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589360c5-21fa-4409-865e-f2313701116f admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:30.288342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 843/1684] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:06:30 2022] PUT /identity/v3/projects/b96e70dba40f475891ea2a40158cbf3e/users/666692a600484d7db9f8e4b3b8427bab/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:06:30.292784 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338cca45-074a-436f-9324-3799ef6bca6c admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:30.315116 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338cca45-074a-436f-9324-3799ef6bca6c admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:30.372935 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8da4a4-e717-4d54-be99-814d7fe00ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:30.373549 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8da4a4-e717-4d54-be99-814d7fe00ba3 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:30.373802 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8da4a4-e717-4d54-be99-814d7fe00ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:30.374069 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8da4a4-e717-4d54-be99-814d7fe00ba3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:30.374733 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8da4a4-e717-4d54-be99-814d7fe00ba3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:30.376319 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8da4a4-e717-4d54-be99-814d7fe00ba3 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:30.385794 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 842/1685] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:30 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 20:06:30.390031 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589360c5-21fa-4409-865e-f2313701116f admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:30.400501 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589360c5-21fa-4409-865e-f2313701116f admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:30.405226 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e797d700-2203-4675-9d20-0947e05c4f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:30.405914 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797d700-2203-4675-9d20-0947e05c4f83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:30.406194 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797d700-2203-4675-9d20-0947e05c4f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:30.406512 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e797d700-2203-4675-9d20-0947e05c4f83 admin admin] PATH_INFO: `/v3/projects/b96e70dba40f475891ea2a40158cbf3e/users/666692a600484d7db9f8e4b3b8427bab/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:30.408446 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e797d700-2203-4675-9d20-0947e05c4f83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b96e70dba40f475891ea2a40158cbf3e, user_id=666692a600484d7db9f8e4b3b8427bab, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:30.411173 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e797d700-2203-4675-9d20-0947e05c4f83 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:30.421879 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 844/1686] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:06:30 2022] PUT /identity/v3/projects/b96e70dba40f475891ea2a40158cbf3e/users/666692a600484d7db9f8e4b3b8427bab/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:30.432789 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276e8e1-1fad-4a6f-89c3-6595d175f2e4 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:30.433110 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276e8e1-1fad-4a6f-89c3-6595d175f2e4 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:30.433414 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7276e8e1-1fad-4a6f-89c3-6595d175f2e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:30.461929 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-7276e8e1-1fad-4a6f-89c3-6595d175f2e4 None None] MFA Rules not processed for user `666692a600484d7db9f8e4b3b8427bab`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:30.463205 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7276e8e1-1fad-4a6f-89c3-6595d175f2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:30.535540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 843/1687] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:30 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:30.546848 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e797d700-2203-4675-9d20-0947e05c4f83 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:30.548561 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-e797d700-2203-4675-9d20-0947e05c4f83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:30.608102 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e797d700-2203-4675-9d20-0947e05c4f83 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:30.611823 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe31e68c-d787-49d6-88fb-addc4a978194 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:30.612440 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe31e68c-d787-49d6-88fb-addc4a978194 service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:30.612722 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe31e68c-d787-49d6-88fb-addc4a978194 service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:30.612997 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe31e68c-d787-49d6-88fb-addc4a978194 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:30.623198 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:30.672131 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe31e68c-d787-49d6-88fb-addc4a978194 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:30.673597 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe31e68c-d787-49d6-88fb-addc4a978194 service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:30.683559 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 845/1688] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:06:30 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:06:34.354368 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7276e8e1-1fad-4a6f-89c3-6595d175f2e4 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:34.363988 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7276e8e1-1fad-4a6f-89c3-6595d175f2e4 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:34.368141 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe303015-515f-4f3e-851f-58e2eb8818af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:34.368756 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe303015-515f-4f3e-851f-58e2eb8818af service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:34.369037 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe303015-515f-4f3e-851f-58e2eb8818af service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:34.369475 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe303015-515f-4f3e-851f-58e2eb8818af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:34.430944 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe303015-515f-4f3e-851f-58e2eb8818af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:34.432303 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe303015-515f-4f3e-851f-58e2eb8818af service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:34.441125 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 844/1689] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:06:34 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:06:35.050691 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe31e68c-d787-49d6-88fb-addc4a978194 service cinder] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:35.062896 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe31e68c-d787-49d6-88fb-addc4a978194 service cinder] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:35.067497 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4242cae0-4bc6-446b-a034-4e89bf100ad2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:35.068112 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242cae0-4bc6-446b-a034-4e89bf100ad2 service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:35.068491 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242cae0-4bc6-446b-a034-4e89bf100ad2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:35.068779 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242cae0-4bc6-446b-a034-4e89bf100ad2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:35.078816 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4242cae0-4bc6-446b-a034-4e89bf100ad2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:35.080651 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4242cae0-4bc6-446b-a034-4e89bf100ad2 service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:35.092620 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 846/1690] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:06:35 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:06:35.293774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe303015-515f-4f3e-851f-58e2eb8818af service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:35.295298 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-fe303015-515f-4f3e-851f-58e2eb8818af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:35.319343 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:35.369618 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe303015-515f-4f3e-851f-58e2eb8818af service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:35.374709 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63fa6c4-2a9e-4846-b05c-092296d0b607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:35.375405 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63fa6c4-2a9e-4846-b05c-092296d0b607 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:35.375683 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63fa6c4-2a9e-4846-b05c-092296d0b607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:35.376118 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63fa6c4-2a9e-4846-b05c-092296d0b607 admin admin] PATH_INFO: `/v3/users/9c6a3f6397c44eeb823df053954900e8` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:35.377354 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63fa6c4-2a9e-4846-b05c-092296d0b607 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c6a3f6397c44eeb823df053954900e8)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:35.379664 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63fa6c4-2a9e-4846-b05c-092296d0b607 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:35.446394 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d63fa6c4-2a9e-4846-b05c-092296d0b607 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c6a3f6397c44eeb823df053954900e8'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:35.475619 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d63fa6c4-2a9e-4846-b05c-092296d0b607 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c6a3f6397c44eeb823df053954900e8'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:35.487653 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d63fa6c4-2a9e-4846-b05c-092296d0b607 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c6a3f6397c44eeb823df053954900e8'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:35.488740 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d63fa6c4-2a9e-4846-b05c-092296d0b607 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c6a3f6397c44eeb823df053954900e8'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:35.496647 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d63fa6c4-2a9e-4846-b05c-092296d0b607 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c6a3f6397c44eeb823df053954900e8'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:35.504969 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 845/1691] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:06:35 2022] DELETE /identity/v3/users/9c6a3f6397c44eeb823df053954900e8 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:35.509901 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5c1da-c813-4734-abbf-97fec5830624 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:35.510931 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5c1da-c813-4734-abbf-97fec5830624 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:35.511286 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d5c1da-c813-4734-abbf-97fec5830624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:35.534776 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-00d5c1da-c813-4734-abbf-97fec5830624 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:35.536194 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-00d5c1da-c813-4734-abbf-97fec5830624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:35.605981 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 847/1692] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:35.617056 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63fa6c4-2a9e-4846-b05c-092296d0b607 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:35.618668 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d63fa6c4-2a9e-4846-b05c-092296d0b607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:35.693928 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63fa6c4-2a9e-4846-b05c-092296d0b607 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:35.698479 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a286b8d3-7eef-474c-96b7-45765384dc7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:35.699117 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286b8d3-7eef-474c-96b7-45765384dc7c service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:35.699433 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286b8d3-7eef-474c-96b7-45765384dc7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:35.700059 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a286b8d3-7eef-474c-96b7-45765384dc7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:35.710517 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a286b8d3-7eef-474c-96b7-45765384dc7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:35.712144 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a286b8d3-7eef-474c-96b7-45765384dc7c service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:35.722918 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 846/1693] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:06:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:06:36.016752 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d5c1da-c813-4734-abbf-97fec5830624 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:36.024373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-00d5c1da-c813-4734-abbf-97fec5830624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:36.037556 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d5c1da-c813-4734-abbf-97fec5830624 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:36.043800 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf29946-3b45-488c-bc49-57ecdd5399bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:36.044472 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf29946-3b45-488c-bc49-57ecdd5399bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:36.044751 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf29946-3b45-488c-bc49-57ecdd5399bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:36.045023 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf29946-3b45-488c-bc49-57ecdd5399bd admin admin] PATH_INFO: `/v3/projects/05112a5d8e634e34a2eed263b6b3db76` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:36.046163 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf29946-3b45-488c-bc49-57ecdd5399bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=05112a5d8e634e34a2eed263b6b3db76)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:36.048259 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf29946-3b45-488c-bc49-57ecdd5399bd admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:36.102556 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-1bf29946-3b45-488c-bc49-57ecdd5399bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05112a5d8e634e34a2eed263b6b3db76', 'request_id': 'req-1bf29946-3b45-488c-bc49-57ecdd5399bd'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:36.111802 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-1bf29946-3b45-488c-bc49-57ecdd5399bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05112a5d8e634e34a2eed263b6b3db76', 'request_id': 'req-1bf29946-3b45-488c-bc49-57ecdd5399bd'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:36.119275 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-1bf29946-3b45-488c-bc49-57ecdd5399bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05112a5d8e634e34a2eed263b6b3db76', 'request_id': 'req-1bf29946-3b45-488c-bc49-57ecdd5399bd'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:36.132272 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-1bf29946-3b45-488c-bc49-57ecdd5399bd admin admin] The token cache is being invalidate because project 05112a5d8e634e34a2eed263b6b3db76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:06:36.132542 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-1bf29946-3b45-488c-bc49-57ecdd5399bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:36.133202 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-1bf29946-3b45-488c-bc49-57ecdd5399bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:36.135123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 848/1694] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:06:36 2022] DELETE /identity/v3/projects/05112a5d8e634e34a2eed263b6b3db76 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:36.140853 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c1df1-85aa-4240-aac9-afd40efac83b None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:36.141201 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c1df1-85aa-4240-aac9-afd40efac83b None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:36.141560 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699c1df1-85aa-4240-aac9-afd40efac83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:36.162038 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-699c1df1-85aa-4240-aac9-afd40efac83b None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:36.163386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-699c1df1-85aa-4240-aac9-afd40efac83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:36.198748 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:36.226206 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 847/1695] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:36.234343 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd93dfe-c426-42af-989b-88665818dc1d None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:36.234704 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd93dfe-c426-42af-989b-88665818dc1d None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:36.235051 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd93dfe-c426-42af-989b-88665818dc1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:36.257309 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-1fd93dfe-c426-42af-989b-88665818dc1d None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:36.258575 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-1fd93dfe-c426-42af-989b-88665818dc1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:36.266423 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 849/1696] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:36.270625 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699c1df1-85aa-4240-aac9-afd40efac83b None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:36.280272 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699c1df1-85aa-4240-aac9-afd40efac83b None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:36.286634 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c227b39-3db5-4e9f-b12d-e4bde669b435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:36.287800 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c227b39-3db5-4e9f-b12d-e4bde669b435 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:36.288078 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c227b39-3db5-4e9f-b12d-e4bde669b435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:36.288389 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c227b39-3db5-4e9f-b12d-e4bde669b435 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:36.288993 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c227b39-3db5-4e9f-b12d-e4bde669b435 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:06:36.289295 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c227b39-3db5-4e9f-b12d-e4bde669b435 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:36.290907 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c227b39-3db5-4e9f-b12d-e4bde669b435 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:36.304423 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 848/1697] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:06:36.308641 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd93dfe-c426-42af-989b-88665818dc1d None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:36.319698 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd93dfe-c426-42af-989b-88665818dc1d None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:36.325224 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1040d-efe4-4acb-95e7-1312882a9cc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:36.325906 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1040d-efe4-4acb-95e7-1312882a9cc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:36.326276 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1040d-efe4-4acb-95e7-1312882a9cc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:36.326744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1040d-efe4-4acb-95e7-1312882a9cc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:36.327525 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b1040d-efe4-4acb-95e7-1312882a9cc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:36.329647 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b1040d-efe4-4acb-95e7-1312882a9cc6 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:36.347043 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:36.364973 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 850/1698] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:06:36 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:06:36.369734 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c227b39-3db5-4e9f-b12d-e4bde669b435 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:36.381779 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c227b39-3db5-4e9f-b12d-e4bde669b435 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:36.453232 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc642d-5ac4-4b75-be81-82275c81cccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:36.453853 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc642d-5ac4-4b75-be81-82275c81cccb admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:36.454135 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc642d-5ac4-4b75-be81-82275c81cccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:36.454446 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebc642d-5ac4-4b75-be81-82275c81cccb admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:36.455084 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc642d-5ac4-4b75-be81-82275c81cccb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:36.457185 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebc642d-5ac4-4b75-be81-82275c81cccb admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:36.489826 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 849/1699] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:06:36 2022] POST /identity/v3/users => generated 293 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:06:36.495833 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1040d-efe4-4acb-95e7-1312882a9cc6 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:36.508387 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1040d-efe4-4acb-95e7-1312882a9cc6 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:36.513615 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc3a2c-47f2-4902-b148-37ad92ad537e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:36.514234 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc3a2c-47f2-4902-b148-37ad92ad537e admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:36.514545 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc3a2c-47f2-4902-b148-37ad92ad537e admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:36.514814 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebc3a2c-47f2-4902-b148-37ad92ad537e admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:36.515771 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebc3a2c-47f2-4902-b148-37ad92ad537e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:36.517060 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebc3a2c-47f2-4902-b148-37ad92ad537e admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:36.529940 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 851/1700] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:06:36.533967 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc642d-5ac4-4b75-be81-82275c81cccb admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:36.546464 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebc642d-5ac4-4b75-be81-82275c81cccb admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:36.566124 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ec0fa-79bb-468e-863b-bfa053246383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:36.566777 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ec0fa-79bb-468e-863b-bfa053246383 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:36.567089 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ec0fa-79bb-468e-863b-bfa053246383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:36.567361 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ec0fa-79bb-468e-863b-bfa053246383 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:36.568011 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8ec0fa-79bb-468e-863b-bfa053246383 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:36.570407 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8ec0fa-79bb-468e-863b-bfa053246383 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:36.581824 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 850/1701] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:36 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:06:36.586191 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc3a2c-47f2-4902-b148-37ad92ad537e admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:36.597085 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebc3a2c-47f2-4902-b148-37ad92ad537e admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:36.602088 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f049555-efb1-41b6-bf76-cac0e064f4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:36.602966 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f049555-efb1-41b6-bf76-cac0e064f4a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:36.603320 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f049555-efb1-41b6-bf76-cac0e064f4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:36.603671 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f049555-efb1-41b6-bf76-cac0e064f4a9 admin admin] PATH_INFO: `/v3/projects/6b4261a7b9e642e5ba063a4790296b66/users/7d0a9c2670794fb4b134e261411fd9cb/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:36.620153 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f049555-efb1-41b6-bf76-cac0e064f4a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b4261a7b9e642e5ba063a4790296b66, user_id=7d0a9c2670794fb4b134e261411fd9cb, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:36.622839 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f049555-efb1-41b6-bf76-cac0e064f4a9 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:36.636535 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 852/1702] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:06:36 2022] PUT /identity/v3/projects/6b4261a7b9e642e5ba063a4790296b66/users/7d0a9c2670794fb4b134e261411fd9cb/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:36.641657 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c60fb6c-6cb3-40b7-b5f9-39c255fb26d1 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:36.641946 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c60fb6c-6cb3-40b7-b5f9-39c255fb26d1 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:36.642268 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c60fb6c-6cb3-40b7-b5f9-39c255fb26d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:36.661029 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-0c60fb6c-6cb3-40b7-b5f9-39c255fb26d1 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:36.662511 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-0c60fb6c-6cb3-40b7-b5f9-39c255fb26d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:36.737738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 851/1703] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:06:36.747658 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f049555-efb1-41b6-bf76-cac0e064f4a9 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:36.749771 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-2f049555-efb1-41b6-bf76-cac0e064f4a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:36.828131 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f049555-efb1-41b6-bf76-cac0e064f4a9 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:36.833015 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e88686-8678-41b1-922a-50e37c1c1d19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:36.834478 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e88686-8678-41b1-922a-50e37c1c1d19 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:36.834583 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e88686-8678-41b1-922a-50e37c1c1d19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:36.834850 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e88686-8678-41b1-922a-50e37c1c1d19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:36.846700 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e88686-8678-41b1-922a-50e37c1c1d19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:36.848144 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e88686-8678-41b1-922a-50e37c1c1d19 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:36.862553 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 853/1704] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:06:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:06:38.338942 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c60fb6c-6cb3-40b7-b5f9-39c255fb26d1 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:38.340522 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-0c60fb6c-6cb3-40b7-b5f9-39c255fb26d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:38.364030 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c60fb6c-6cb3-40b7-b5f9-39c255fb26d1 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:38.374491 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d57f0a-3950-46db-ace0-f4b42b64fb7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:38.376821 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d57f0a-3950-46db-ace0-f4b42b64fb7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:38.377268 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d57f0a-3950-46db-ace0-f4b42b64fb7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:38.377933 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d57f0a-3950-46db-ace0-f4b42b64fb7d admin admin] PATH_INFO: `/v3/users/072140f112234d5295fead08bde4185c` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:38.379357 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d57f0a-3950-46db-ace0-f4b42b64fb7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=072140f112234d5295fead08bde4185c)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:38.381797 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d57f0a-3950-46db-ace0-f4b42b64fb7d admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:38.459187 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:38.466567 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-98d57f0a-3950-46db-ace0-f4b42b64fb7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '072140f112234d5295fead08bde4185c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:38.488202 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-98d57f0a-3950-46db-ace0-f4b42b64fb7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '072140f112234d5295fead08bde4185c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:38.500076 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-98d57f0a-3950-46db-ace0-f4b42b64fb7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '072140f112234d5295fead08bde4185c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:38.501356 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-98d57f0a-3950-46db-ace0-f4b42b64fb7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '072140f112234d5295fead08bde4185c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:38.514579 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-98d57f0a-3950-46db-ace0-f4b42b64fb7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '072140f112234d5295fead08bde4185c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:38.526596 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 852/1705] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:06:38 2022] DELETE /identity/v3/users/072140f112234d5295fead08bde4185c => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:38.531528 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55474bce-578b-4b8c-a17a-1bf280188229 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:38.531528 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55474bce-578b-4b8c-a17a-1bf280188229 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:38.531528 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55474bce-578b-4b8c-a17a-1bf280188229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:38.562785 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-55474bce-578b-4b8c-a17a-1bf280188229 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:38.564093 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-55474bce-578b-4b8c-a17a-1bf280188229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:38.697169 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 854/1706] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:06:38.703405 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d57f0a-3950-46db-ace0-f4b42b64fb7d admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:38.705178 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-98d57f0a-3950-46db-ace0-f4b42b64fb7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:38.810037 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d57f0a-3950-46db-ace0-f4b42b64fb7d admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:38.814185 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b935c68-5996-4a4f-9c67-72f4141badf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:38.814829 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b935c68-5996-4a4f-9c67-72f4141badf0 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:38.815244 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b935c68-5996-4a4f-9c67-72f4141badf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:38.815593 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b935c68-5996-4a4f-9c67-72f4141badf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:38.826974 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b935c68-5996-4a4f-9c67-72f4141badf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:38.828414 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b935c68-5996-4a4f-9c67-72f4141badf0 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:38.839784 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 853/1707] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:06:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 20:06:39.179314 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55474bce-578b-4b8c-a17a-1bf280188229 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:39.180798 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-55474bce-578b-4b8c-a17a-1bf280188229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:39.203297 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55474bce-578b-4b8c-a17a-1bf280188229 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:39.214621 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7018879a-ac54-464e-aaf2-b847fd17db6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:39.215334 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7018879a-ac54-464e-aaf2-b847fd17db6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:39.215621 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7018879a-ac54-464e-aaf2-b847fd17db6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:39.215944 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7018879a-ac54-464e-aaf2-b847fd17db6c admin admin] PATH_INFO: `/v3/projects/8ecc025a0a434b69863a51807c7fa078` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:39.230469 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7018879a-ac54-464e-aaf2-b847fd17db6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ecc025a0a434b69863a51807c7fa078)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:39.237268 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7018879a-ac54-464e-aaf2-b847fd17db6c admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:39.334004 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-7018879a-ac54-464e-aaf2-b847fd17db6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ecc025a0a434b69863a51807c7fa078', 'request_id': 'req-7018879a-ac54-464e-aaf2-b847fd17db6c'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:39.355500 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-7018879a-ac54-464e-aaf2-b847fd17db6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ecc025a0a434b69863a51807c7fa078', 'request_id': 'req-7018879a-ac54-464e-aaf2-b847fd17db6c'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:39.357200 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:INSERT=3,keystone:SELECT=92,keystone:DELETE=5 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:39.363742 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-7018879a-ac54-464e-aaf2-b847fd17db6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ecc025a0a434b69863a51807c7fa078', 'request_id': 'req-7018879a-ac54-464e-aaf2-b847fd17db6c'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:39.379395 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-7018879a-ac54-464e-aaf2-b847fd17db6c admin admin] The token cache is being invalidate because project 8ecc025a0a434b69863a51807c7fa078 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:06:39.379700 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-7018879a-ac54-464e-aaf2-b847fd17db6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:39.380456 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-7018879a-ac54-464e-aaf2-b847fd17db6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:39.382541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 855/1708] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:06:39 2022] DELETE /identity/v3/projects/8ecc025a0a434b69863a51807c7fa078 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:39.386857 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b935c68-5996-4a4f-9c67-72f4141badf0 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:39.398135 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b935c68-5996-4a4f-9c67-72f4141badf0 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:39.486007 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c4b16-bf5e-4d77-be82-96d538fae530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:39.486859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c4b16-bf5e-4d77-be82-96d538fae530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:39.487243 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c4b16-bf5e-4d77-be82-96d538fae530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:39.488119 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c4b16-bf5e-4d77-be82-96d538fae530 admin admin] PATH_INFO: `/v3/users/e8eff3a9281744b69ab3ef4e2bc0442f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:39.490118 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c4b16-bf5e-4d77-be82-96d538fae530 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8eff3a9281744b69ab3ef4e2bc0442f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:39.494851 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c4b16-bf5e-4d77-be82-96d538fae530 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:39.562009 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-132c4b16-bf5e-4d77-be82-96d538fae530 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8eff3a9281744b69ab3ef4e2bc0442f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:39.576850 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:39.578397 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-132c4b16-bf5e-4d77-be82-96d538fae530 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8eff3a9281744b69ab3ef4e2bc0442f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:39.602935 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-132c4b16-bf5e-4d77-be82-96d538fae530 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8eff3a9281744b69ab3ef4e2bc0442f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:39.604430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-132c4b16-bf5e-4d77-be82-96d538fae530 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8eff3a9281744b69ab3ef4e2bc0442f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:39.615924 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-132c4b16-bf5e-4d77-be82-96d538fae530 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8eff3a9281744b69ab3ef4e2bc0442f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:39.626628 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 854/1709] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:06:39 2022] DELETE /identity/v3/users/e8eff3a9281744b69ab3ef4e2bc0442f => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:39.964012 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7018879a-ac54-464e-aaf2-b847fd17db6c admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:39.966341 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-7018879a-ac54-464e-aaf2-b847fd17db6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:39.991347 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7018879a-ac54-464e-aaf2-b847fd17db6c admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:40.073175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563d1e2-e4cf-4af0-adb7-5d6f39b82627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:40.074101 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563d1e2-e4cf-4af0-adb7-5d6f39b82627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:40.074417 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563d1e2-e4cf-4af0-adb7-5d6f39b82627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:40.075043 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563d1e2-e4cf-4af0-adb7-5d6f39b82627 admin admin] PATH_INFO: `/v3/projects/6a65d3b182284519bb1c2af56915381a` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:40.076381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563d1e2-e4cf-4af0-adb7-5d6f39b82627 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6a65d3b182284519bb1c2af56915381a)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:40.079804 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563d1e2-e4cf-4af0-adb7-5d6f39b82627 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:40.137416 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-5563d1e2-e4cf-4af0-adb7-5d6f39b82627 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6a65d3b182284519bb1c2af56915381a', 'request_id': 'req-5563d1e2-e4cf-4af0-adb7-5d6f39b82627'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:40.148039 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-5563d1e2-e4cf-4af0-adb7-5d6f39b82627 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6a65d3b182284519bb1c2af56915381a', 'request_id': 'req-5563d1e2-e4cf-4af0-adb7-5d6f39b82627'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:40.155315 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-5563d1e2-e4cf-4af0-adb7-5d6f39b82627 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6a65d3b182284519bb1c2af56915381a', 'request_id': 'req-5563d1e2-e4cf-4af0-adb7-5d6f39b82627'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:40.171788 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-5563d1e2-e4cf-4af0-adb7-5d6f39b82627 admin admin] The token cache is being invalidate because project 6a65d3b182284519bb1c2af56915381a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:06:40.172449 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-5563d1e2-e4cf-4af0-adb7-5d6f39b82627 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:40.173549 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-5563d1e2-e4cf-4af0-adb7-5d6f39b82627 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:40.179099 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 856/1710] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:06:39 2022] DELETE /identity/v3/projects/6a65d3b182284519bb1c2af56915381a => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:40.180827 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56bb5f4-185c-4dfe-bc0e-f7e6434e79d3 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:40.181118 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56bb5f4-185c-4dfe-bc0e-f7e6434e79d3 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:40.181397 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56bb5f4-185c-4dfe-bc0e-f7e6434e79d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:40.210545 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-e56bb5f4-185c-4dfe-bc0e-f7e6434e79d3 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:40.212102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e56bb5f4-185c-4dfe-bc0e-f7e6434e79d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:40.299058 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 855/1711] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:40.306820 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6b81e-2721-456b-91c4-02f1fa690ea9 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:40.307186 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6b81e-2721-456b-91c4-02f1fa690ea9 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:40.307530 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e6b81e-2721-456b-91c4-02f1fa690ea9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:40.334867 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-86e6b81e-2721-456b-91c4-02f1fa690ea9 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:40.339120 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-86e6b81e-2721-456b-91c4-02f1fa690ea9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:40.358029 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56bb5f4-185c-4dfe-bc0e-f7e6434e79d3 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:40.367757 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 857/1712] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:40.372240 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56bb5f4-185c-4dfe-bc0e-f7e6434e79d3 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:40.380218 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a06689-a13d-4cb6-b52f-ca0841e647c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:40.381767 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a06689-a13d-4cb6-b52f-ca0841e647c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:40.382060 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a06689-a13d-4cb6-b52f-ca0841e647c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:40.382364 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a06689-a13d-4cb6-b52f-ca0841e647c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:40.383863 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a06689-a13d-4cb6-b52f-ca0841e647c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:06:40.384177 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a06689-a13d-4cb6-b52f-ca0841e647c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:40.386068 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a06689-a13d-4cb6-b52f-ca0841e647c4 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:40.416023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 856/1713] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:06:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:06:40.420552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6b81e-2721-456b-91c4-02f1fa690ea9 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:40.430840 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e6b81e-2721-456b-91c4-02f1fa690ea9 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:40.436386 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b9ce05-d5b7-48c2-8979-69e7c975d070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:40.437020 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9ce05-d5b7-48c2-8979-69e7c975d070 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:40.437279 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9ce05-d5b7-48c2-8979-69e7c975d070 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:40.437553 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b9ce05-d5b7-48c2-8979-69e7c975d070 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:40.438349 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b9ce05-d5b7-48c2-8979-69e7c975d070 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:40.440919 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b9ce05-d5b7-48c2-8979-69e7c975d070 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:40.475762 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 858/1714] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:06:40 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:06:40.477281 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a06689-a13d-4cb6-b52f-ca0841e647c4 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:40.494547 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a06689-a13d-4cb6-b52f-ca0841e647c4 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:40.557224 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f00f0-20c7-4240-9c10-66a5a1f5e673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:40.558069 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f00f0-20c7-4240-9c10-66a5a1f5e673 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:40.558466 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f00f0-20c7-4240-9c10-66a5a1f5e673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:40.558855 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29f00f0-20c7-4240-9c10-66a5a1f5e673 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:40.559592 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f00f0-20c7-4240-9c10-66a5a1f5e673 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:40.561597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29f00f0-20c7-4240-9c10-66a5a1f5e673 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:40.585171 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 857/1715] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:06:40 2022] POST /identity/v3/users => generated 283 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:06:40.589895 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b9ce05-d5b7-48c2-8979-69e7c975d070 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:40.601443 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b9ce05-d5b7-48c2-8979-69e7c975d070 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:40.606084 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20dc567-4484-42e2-99b2-684fb2dc7231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:40.606744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20dc567-4484-42e2-99b2-684fb2dc7231 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:40.607027 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20dc567-4484-42e2-99b2-684fb2dc7231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:40.607298 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20dc567-4484-42e2-99b2-684fb2dc7231 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:40.607952 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20dc567-4484-42e2-99b2-684fb2dc7231 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:40.609872 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20dc567-4484-42e2-99b2-684fb2dc7231 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:40.623759 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 859/1716] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:06:40.627260 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f00f0-20c7-4240-9c10-66a5a1f5e673 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:40.643071 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29f00f0-20c7-4240-9c10-66a5a1f5e673 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:40.648448 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3fa00a-3773-49e6-8819-66e25c9b55a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:40.649076 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fa00a-3773-49e6-8819-66e25c9b55a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:40.649949 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fa00a-3773-49e6-8819-66e25c9b55a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:40.649949 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3fa00a-3773-49e6-8819-66e25c9b55a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:40.650346 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3fa00a-3773-49e6-8819-66e25c9b55a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:40.651991 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3fa00a-3773-49e6-8819-66e25c9b55a8 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:40.664783 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 858/1717] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:06:40.668840 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20dc567-4484-42e2-99b2-684fb2dc7231 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:40.681636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20dc567-4484-42e2-99b2-684fb2dc7231 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:40.686745 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34991e48-eb2e-4a82-be80-95e7daf93592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:40.687545 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34991e48-eb2e-4a82-be80-95e7daf93592 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:40.687837 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34991e48-eb2e-4a82-be80-95e7daf93592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:40.688127 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34991e48-eb2e-4a82-be80-95e7daf93592 admin admin] PATH_INFO: `/v3/projects/3660c6fffa274060ada1984b56df5578/users/3388d85f2b3c477d8e392a3c6ce47f11/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:40.710618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34991e48-eb2e-4a82-be80-95e7daf93592 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3660c6fffa274060ada1984b56df5578, user_id=3388d85f2b3c477d8e392a3c6ce47f11, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:40.715084 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34991e48-eb2e-4a82-be80-95e7daf93592 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:40.742624 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 860/1718] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:06:40 2022] PUT /identity/v3/projects/3660c6fffa274060ada1984b56df5578/users/3388d85f2b3c477d8e392a3c6ce47f11/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:40.753970 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364f341-2e2c-4c79-b7ec-1274bb79877a None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:40.754970 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364f341-2e2c-4c79-b7ec-1274bb79877a None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:40.754970 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4364f341-2e2c-4c79-b7ec-1274bb79877a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:40.800666 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-4364f341-2e2c-4c79-b7ec-1274bb79877a None None] MFA Rules not processed for user `3388d85f2b3c477d8e392a3c6ce47f11`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:40.801918 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4364f341-2e2c-4c79-b7ec-1274bb79877a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:40.886494 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:40.949164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 859/1719] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:40.953571 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34991e48-eb2e-4a82-be80-95e7daf93592 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:40.972820 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:41.058093 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34991e48-eb2e-4a82-be80-95e7daf93592 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:41.070057 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01c36d-c3d4-42ac-af6c-f94093235b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:41.071012 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01c36d-c3d4-42ac-af6c-f94093235b2e admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:41.071440 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01c36d-c3d4-42ac-af6c-f94093235b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:41.072205 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e01c36d-c3d4-42ac-af6c-f94093235b2e admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:41.073037 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01c36d-c3d4-42ac-af6c-f94093235b2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:41.075905 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e01c36d-c3d4-42ac-af6c-f94093235b2e admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:41.127894 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 861/1720] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:06:40 2022] POST /identity/v3/projects => generated 365 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:06:41.129923 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4364f341-2e2c-4c79-b7ec-1274bb79877a None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:41.154475 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4364f341-2e2c-4c79-b7ec-1274bb79877a None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:41.269533 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35605f-a1d9-4fa3-ab38-d0ed7f636d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:41.270163 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35605f-a1d9-4fa3-ab38-d0ed7f636d22 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:41.270478 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35605f-a1d9-4fa3-ab38-d0ed7f636d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:41.271859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce35605f-a1d9-4fa3-ab38-d0ed7f636d22 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:41.272533 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35605f-a1d9-4fa3-ab38-d0ed7f636d22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:41.276026 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce35605f-a1d9-4fa3-ab38-d0ed7f636d22 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:41.308889 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01c36d-c3d4-42ac-af6c-f94093235b2e admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:41.315637 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 860/1721] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:06:41 2022] POST /identity/v3/users => generated 289 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:06:41.324334 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e01c36d-c3d4-42ac-af6c-f94093235b2e admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:41.337703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa6cfe-da77-4cfc-aa34-ecae6e21d410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:41.338363 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa6cfe-da77-4cfc-aa34-ecae6e21d410 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:41.338765 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa6cfe-da77-4cfc-aa34-ecae6e21d410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:41.338970 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa6cfe-da77-4cfc-aa34-ecae6e21d410 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:41.339558 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aa6cfe-da77-4cfc-aa34-ecae6e21d410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:41.341156 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aa6cfe-da77-4cfc-aa34-ecae6e21d410 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:41.358356 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 862/1722] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:06:41.360908 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35605f-a1d9-4fa3-ab38-d0ed7f636d22 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:41.374821 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce35605f-a1d9-4fa3-ab38-d0ed7f636d22 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:41.379815 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800ab40-c128-423f-bc8a-9f7ef4243c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:41.380752 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800ab40-c128-423f-bc8a-9f7ef4243c80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:41.381197 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800ab40-c128-423f-bc8a-9f7ef4243c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:41.381482 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c800ab40-c128-423f-bc8a-9f7ef4243c80 admin admin] PATH_INFO: `/v3/projects/59bf64a6275a44599b7772f80c482626/users/a412e296ea9c4753adf440b9fc5e356a/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:41.418624 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c800ab40-c128-423f-bc8a-9f7ef4243c80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59bf64a6275a44599b7772f80c482626, user_id=a412e296ea9c4753adf440b9fc5e356a, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:41.434384 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c800ab40-c128-423f-bc8a-9f7ef4243c80 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:41.453270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 861/1723] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:06:41 2022] PUT /identity/v3/projects/59bf64a6275a44599b7772f80c482626/users/a412e296ea9c4753adf440b9fc5e356a/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:41.457204 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa6cfe-da77-4cfc-aa34-ecae6e21d410 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:41.469771 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa6cfe-da77-4cfc-aa34-ecae6e21d410 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:41.553268 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8afde6-4108-4290-a401-a5f7a6fe0624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:41.553901 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8afde6-4108-4290-a401-a5f7a6fe0624 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:41.554187 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8afde6-4108-4290-a401-a5f7a6fe0624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:41.554487 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8afde6-4108-4290-a401-a5f7a6fe0624 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:41.555145 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8afde6-4108-4290-a401-a5f7a6fe0624 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:41.558923 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8afde6-4108-4290-a401-a5f7a6fe0624 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:41.572913 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 863/1724] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:41 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:06:41.576791 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800ab40-c128-423f-bc8a-9f7ef4243c80 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:41.594829 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c800ab40-c128-423f-bc8a-9f7ef4243c80 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:41.619470 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5203284-4447-4a75-b1f9-dc19a1ac2fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:41.620166 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5203284-4447-4a75-b1f9-dc19a1ac2fbf admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:41.620445 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5203284-4447-4a75-b1f9-dc19a1ac2fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:41.620718 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5203284-4447-4a75-b1f9-dc19a1ac2fbf admin admin] PATH_INFO: `/v3/projects/59bf64a6275a44599b7772f80c482626/users/a412e296ea9c4753adf440b9fc5e356a/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:41.623382 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5203284-4447-4a75-b1f9-dc19a1ac2fbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=59bf64a6275a44599b7772f80c482626, user_id=a412e296ea9c4753adf440b9fc5e356a, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:41.625705 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5203284-4447-4a75-b1f9-dc19a1ac2fbf admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:41.640040 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 862/1725] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:06:41 2022] PUT /identity/v3/projects/59bf64a6275a44599b7772f80c482626/users/a412e296ea9c4753adf440b9fc5e356a/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:06:41.650198 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75e0f4-4ffa-4239-b1c6-5a9937846160 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:41.651297 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75e0f4-4ffa-4239-b1c6-5a9937846160 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:41.651828 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a75e0f4-4ffa-4239-b1c6-5a9937846160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:41.687242 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-7a75e0f4-4ffa-4239-b1c6-5a9937846160 None None] MFA Rules not processed for user `a412e296ea9c4753adf440b9fc5e356a`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:41.688534 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-7a75e0f4-4ffa-4239-b1c6-5a9937846160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:41.774596 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 864/1726] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:41 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:41.783597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5203284-4447-4a75-b1f9-dc19a1ac2fbf admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:41.785392 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-b5203284-4447-4a75-b1f9-dc19a1ac2fbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:41.884118 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5203284-4447-4a75-b1f9-dc19a1ac2fbf admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:41.895240 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c322382b-05a1-4694-8c88-78c752d87a6a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:41.898551 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322382b-05a1-4694-8c88-78c752d87a6a service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:41.899022 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322382b-05a1-4694-8c88-78c752d87a6a service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:41.899871 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c322382b-05a1-4694-8c88-78c752d87a6a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:41.948435 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c322382b-05a1-4694-8c88-78c752d87a6a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:41.950475 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c322382b-05a1-4694-8c88-78c752d87a6a service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:41.975428 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 863/1727] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:06:41 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:06:42.112607 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75e0f4-4ffa-4239-b1c6-5a9937846160 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:42.125106 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a75e0f4-4ffa-4239-b1c6-5a9937846160 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:42.144883 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49056266-a3b0-410c-82ff-ef1b65da315c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:42.145536 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49056266-a3b0-410c-82ff-ef1b65da315c service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:42.145902 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49056266-a3b0-410c-82ff-ef1b65da315c service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:42.146720 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49056266-a3b0-410c-82ff-ef1b65da315c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:42.161700 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:42.276482 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49056266-a3b0-410c-82ff-ef1b65da315c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:42.277936 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49056266-a3b0-410c-82ff-ef1b65da315c service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:42.288813 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 865/1728] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:06:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:06:42.407195 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829bef8-09b7-4a39-abff-2012343faf9a None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:42.407719 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829bef8-09b7-4a39-abff-2012343faf9a None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:42.408189 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a829bef8-09b7-4a39-abff-2012343faf9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:42.459623 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-a829bef8-09b7-4a39-abff-2012343faf9a None None] MFA Rules not processed for user `7d0a9c2670794fb4b134e261411fd9cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:42.461198 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a829bef8-09b7-4a39-abff-2012343faf9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:42.518329 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:42.660393 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 864/1729] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:42 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:42.664373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49056266-a3b0-410c-82ff-ef1b65da315c service cinder] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:42.666099 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-49056266-a3b0-410c-82ff-ef1b65da315c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:42.676927 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829bef8-09b7-4a39-abff-2012343faf9a None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:42.685711 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a829bef8-09b7-4a39-abff-2012343faf9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:42.823113 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49056266-a3b0-410c-82ff-ef1b65da315c service cinder] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:42.828222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762bd479-b82e-408b-ab88-e65a7197556a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:42.828770 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bd479-b82e-408b-ab88-e65a7197556a admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:42.829206 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bd479-b82e-408b-ab88-e65a7197556a admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:42.829909 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762bd479-b82e-408b-ab88-e65a7197556a admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:42.830133 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762bd479-b82e-408b-ab88-e65a7197556a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:42.832473 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762bd479-b82e-408b-ab88-e65a7197556a admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:42.845817 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a829bef8-09b7-4a39-abff-2012343faf9a None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:42.851525 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a24b27a-8fbd-48ed-969e-a711afa80ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:42.852276 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a24b27a-8fbd-48ed-969e-a711afa80ef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:42.852632 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a24b27a-8fbd-48ed-969e-a711afa80ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:42.853109 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a24b27a-8fbd-48ed-969e-a711afa80ef0 admin admin] PATH_INFO: `/v3/users/3b32d4fd7c8740229cacf6b5c90ac781` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:42.854369 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a24b27a-8fbd-48ed-969e-a711afa80ef0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b32d4fd7c8740229cacf6b5c90ac781)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:42.856689 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a24b27a-8fbd-48ed-969e-a711afa80ef0 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:42.925582 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 866/1730] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:06:42 2022] POST /identity/v3/projects => generated 383 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:06:42.927328 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762bd479-b82e-408b-ab88-e65a7197556a admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:42.947298 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-3a24b27a-8fbd-48ed-969e-a711afa80ef0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b32d4fd7c8740229cacf6b5c90ac781'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:42.964066 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-3a24b27a-8fbd-48ed-969e-a711afa80ef0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b32d4fd7c8740229cacf6b5c90ac781'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:42.974684 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-3a24b27a-8fbd-48ed-969e-a711afa80ef0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b32d4fd7c8740229cacf6b5c90ac781'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:42.976337 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-3a24b27a-8fbd-48ed-969e-a711afa80ef0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b32d4fd7c8740229cacf6b5c90ac781'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:42.992454 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-3a24b27a-8fbd-48ed-969e-a711afa80ef0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b32d4fd7c8740229cacf6b5c90ac781'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:43.003074 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 865/1731] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:06:42 2022] DELETE /identity/v3/users/3b32d4fd7c8740229cacf6b5c90ac781 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:06:43.007846 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069bd77-dfc4-4f0e-b0a3-8ca5de8b12c1 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:43.008129 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069bd77-dfc4-4f0e-b0a3-8ca5de8b12c1 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:43.010599 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5069bd77-dfc4-4f0e-b0a3-8ca5de8b12c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:43.053176 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-5069bd77-dfc4-4f0e-b0a3-8ca5de8b12c1 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:43.054600 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5069bd77-dfc4-4f0e-b0a3-8ca5de8b12c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:43.078557 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762bd479-b82e-408b-ab88-e65a7197556a admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:43.104408 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:43.126949 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:43.148841 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 866/1732] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:43.158084 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069bd77-dfc4-4f0e-b0a3-8ca5de8b12c1 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:43.160372 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5069bd77-dfc4-4f0e-b0a3-8ca5de8b12c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:43.171843 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb68815b-e543-4aec-b7dc-85a893e537fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:43.173677 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68815b-e543-4aec-b7dc-85a893e537fa admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:43.173961 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68815b-e543-4aec-b7dc-85a893e537fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:43.174236 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68815b-e543-4aec-b7dc-85a893e537fa admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:43.178260 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb68815b-e543-4aec-b7dc-85a893e537fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:43.182173 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb68815b-e543-4aec-b7dc-85a893e537fa admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:43.245905 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 867/1733] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:06:42 2022] POST /identity/v3/users => generated 298 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:06:43.249377 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb68815b-e543-4aec-b7dc-85a893e537fa admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:43.254496 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-fb68815b-e543-4aec-b7dc-85a893e537fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:43.282590 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb68815b-e543-4aec-b7dc-85a893e537fa admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:43.298182 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1f1c7-be6f-4d22-b420-260dde0dc4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:43.299364 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1f1c7-be6f-4d22-b420-260dde0dc4d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:43.301296 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1f1c7-be6f-4d22-b420-260dde0dc4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:43.301821 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1f1c7-be6f-4d22-b420-260dde0dc4d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:43.303292 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a1f1c7-be6f-4d22-b420-260dde0dc4d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:43.308156 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a1f1c7-be6f-4d22-b420-260dde0dc4d4 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:43.323731 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 868/1734] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:06:43.325003 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5069bd77-dfc4-4f0e-b0a3-8ca5de8b12c1 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:43.327761 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1f1c7-be6f-4d22-b420-260dde0dc4d4 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:43.333136 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf35d39-f03f-4465-9106-57b15d5c46c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:43.334328 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf35d39-f03f-4465-9106-57b15d5c46c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:43.334620 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf35d39-f03f-4465-9106-57b15d5c46c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:43.334974 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf35d39-f03f-4465-9106-57b15d5c46c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:43.348095 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf35d39-f03f-4465-9106-57b15d5c46c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:43.348747 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1f1c7-be6f-4d22-b420-260dde0dc4d4 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:43.350198 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf35d39-f03f-4465-9106-57b15d5c46c2 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:43.361703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34b0d5-55cb-4c91-adef-efe72c9fbf4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:43.362687 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34b0d5-55cb-4c91-adef-efe72c9fbf4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:43.363083 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34b0d5-55cb-4c91-adef-efe72c9fbf4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:43.363535 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa34b0d5-55cb-4c91-adef-efe72c9fbf4b admin admin] PATH_INFO: `/v3/projects/6f612c9285bd4cd9b00a0092f350d1d8/users/25cd9cc7253a4932ad013f1188995332/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:43.372984 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 867/1735] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:06:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 20:06:43.409360 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa34b0d5-55cb-4c91-adef-efe72c9fbf4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f612c9285bd4cd9b00a0092f350d1d8, user_id=25cd9cc7253a4932ad013f1188995332, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:43.414574 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa34b0d5-55cb-4c91-adef-efe72c9fbf4b admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:43.442108 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 869/1736] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:06:43 2022] PUT /identity/v3/projects/6f612c9285bd4cd9b00a0092f350d1d8/users/25cd9cc7253a4932ad013f1188995332/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:43.446922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf35d39-f03f-4465-9106-57b15d5c46c2 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:43.461584 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf35d39-f03f-4465-9106-57b15d5c46c2 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:43.573010 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add318a6-46fa-4d50-ae7c-c63c4e81a294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:43.574697 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add318a6-46fa-4d50-ae7c-c63c4e81a294 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:43.575084 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add318a6-46fa-4d50-ae7c-c63c4e81a294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:43.575507 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add318a6-46fa-4d50-ae7c-c63c4e81a294 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:43.576632 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add318a6-46fa-4d50-ae7c-c63c4e81a294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:43.578197 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add318a6-46fa-4d50-ae7c-c63c4e81a294 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:43.597507 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 868/1737] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:43 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 20:06:43.598808 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34b0d5-55cb-4c91-adef-efe72c9fbf4b admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:43.619538 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa34b0d5-55cb-4c91-adef-efe72c9fbf4b admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:43.633359 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9336dd-cd48-4352-ae3e-ac300b8b2bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:43.634202 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9336dd-cd48-4352-ae3e-ac300b8b2bf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:43.634639 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9336dd-cd48-4352-ae3e-ac300b8b2bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:43.635007 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9336dd-cd48-4352-ae3e-ac300b8b2bf7 admin admin] PATH_INFO: `/v3/projects/6f612c9285bd4cd9b00a0092f350d1d8/users/25cd9cc7253a4932ad013f1188995332/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:43.638866 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9336dd-cd48-4352-ae3e-ac300b8b2bf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f612c9285bd4cd9b00a0092f350d1d8, user_id=25cd9cc7253a4932ad013f1188995332, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:43.643666 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9336dd-cd48-4352-ae3e-ac300b8b2bf7 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:43.668427 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 870/1738] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:06:43 2022] PUT /identity/v3/projects/6f612c9285bd4cd9b00a0092f350d1d8/users/25cd9cc7253a4932ad013f1188995332/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:43.801356 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add318a6-46fa-4d50-ae7c-c63c4e81a294 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:43.803679 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-add318a6-46fa-4d50-ae7c-c63c4e81a294 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:43.821697 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add318a6-46fa-4d50-ae7c-c63c4e81a294 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:43.910471 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada6b312-99f9-48e4-b09f-47f3a338d4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:43.910471 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada6b312-99f9-48e4-b09f-47f3a338d4ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:43.910471 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada6b312-99f9-48e4-b09f-47f3a338d4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:43.910471 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada6b312-99f9-48e4-b09f-47f3a338d4ed admin admin] PATH_INFO: `/v3/projects/47032be45c734a00973adcbfa5696c37` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:43.912715 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada6b312-99f9-48e4-b09f-47f3a338d4ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=47032be45c734a00973adcbfa5696c37)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:43.915165 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada6b312-99f9-48e4-b09f-47f3a338d4ed admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:43.946568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:43.999805 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-ada6b312-99f9-48e4-b09f-47f3a338d4ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47032be45c734a00973adcbfa5696c37', 'request_id': 'req-ada6b312-99f9-48e4-b09f-47f3a338d4ed'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:44.056287 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-ada6b312-99f9-48e4-b09f-47f3a338d4ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47032be45c734a00973adcbfa5696c37', 'request_id': 'req-ada6b312-99f9-48e4-b09f-47f3a338d4ed'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:44.066415 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-ada6b312-99f9-48e4-b09f-47f3a338d4ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47032be45c734a00973adcbfa5696c37', 'request_id': 'req-ada6b312-99f9-48e4-b09f-47f3a338d4ed'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:44.080913 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-ada6b312-99f9-48e4-b09f-47f3a338d4ed admin admin] The token cache is being invalidate because project 47032be45c734a00973adcbfa5696c37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:06:44.080913 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-ada6b312-99f9-48e4-b09f-47f3a338d4ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:44.081609 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-ada6b312-99f9-48e4-b09f-47f3a338d4ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:44.083613 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 869/1739] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:06:43 2022] DELETE /identity/v3/projects/47032be45c734a00973adcbfa5696c37 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:44.087735 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9336dd-cd48-4352-ae3e-ac300b8b2bf7 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:44.089788 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-3f9336dd-cd48-4352-ae3e-ac300b8b2bf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:44.114677 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9336dd-cd48-4352-ae3e-ac300b8b2bf7 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:44.292055 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a895a2-105b-40b2-bb44-fa2251d17d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:44.294620 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a895a2-105b-40b2-bb44-fa2251d17d2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:44.296072 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a895a2-105b-40b2-bb44-fa2251d17d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:44.296636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a895a2-105b-40b2-bb44-fa2251d17d2f admin admin] PATH_INFO: `/v3/users/59735e50eea34cc398abd28921df601e` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:44.299219 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a895a2-105b-40b2-bb44-fa2251d17d2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=59735e50eea34cc398abd28921df601e)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:44.301373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a895a2-105b-40b2-bb44-fa2251d17d2f admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:44.458444 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-e8a895a2-105b-40b2-bb44-fa2251d17d2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59735e50eea34cc398abd28921df601e'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:44.477556 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-e8a895a2-105b-40b2-bb44-fa2251d17d2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59735e50eea34cc398abd28921df601e'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:44.488574 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-e8a895a2-105b-40b2-bb44-fa2251d17d2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59735e50eea34cc398abd28921df601e'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:44.506552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-e8a895a2-105b-40b2-bb44-fa2251d17d2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59735e50eea34cc398abd28921df601e'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:44.507919 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-e8a895a2-105b-40b2-bb44-fa2251d17d2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59735e50eea34cc398abd28921df601e'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:44.511395 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:44.523957 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 871/1740] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:06:44 2022] DELETE /identity/v3/users/59735e50eea34cc398abd28921df601e => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:44.894734 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada6b312-99f9-48e4-b09f-47f3a338d4ed admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:44.910711 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada6b312-99f9-48e4-b09f-47f3a338d4ed admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:45.064875 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022acc2b-f361-4dc6-b3dc-fe1dbefa8b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:45.066105 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022acc2b-f361-4dc6-b3dc-fe1dbefa8b5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:45.066626 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022acc2b-f361-4dc6-b3dc-fe1dbefa8b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:45.067264 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022acc2b-f361-4dc6-b3dc-fe1dbefa8b5d admin admin] PATH_INFO: `/v3/projects/200d1d7a5f4b4f4a9ecdda09cf8010a7` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:45.069134 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022acc2b-f361-4dc6-b3dc-fe1dbefa8b5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=200d1d7a5f4b4f4a9ecdda09cf8010a7)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:45.072479 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022acc2b-f361-4dc6-b3dc-fe1dbefa8b5d admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:45.165488 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-022acc2b-f361-4dc6-b3dc-fe1dbefa8b5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '200d1d7a5f4b4f4a9ecdda09cf8010a7', 'request_id': 'req-022acc2b-f361-4dc6-b3dc-fe1dbefa8b5d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:45.176333 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-022acc2b-f361-4dc6-b3dc-fe1dbefa8b5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '200d1d7a5f4b4f4a9ecdda09cf8010a7', 'request_id': 'req-022acc2b-f361-4dc6-b3dc-fe1dbefa8b5d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:45.183906 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-022acc2b-f361-4dc6-b3dc-fe1dbefa8b5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '200d1d7a5f4b4f4a9ecdda09cf8010a7', 'request_id': 'req-022acc2b-f361-4dc6-b3dc-fe1dbefa8b5d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:45.200089 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-022acc2b-f361-4dc6-b3dc-fe1dbefa8b5d admin admin] The token cache is being invalidate because project 200d1d7a5f4b4f4a9ecdda09cf8010a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:06:45.200382 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-022acc2b-f361-4dc6-b3dc-fe1dbefa8b5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:45.201076 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-022acc2b-f361-4dc6-b3dc-fe1dbefa8b5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:45.206311 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 870/1741] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:06:44 2022] DELETE /identity/v3/projects/200d1d7a5f4b4f4a9ecdda09cf8010a7 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:45.207487 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a895a2-105b-40b2-bb44-fa2251d17d2f admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:45.209464 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-e8a895a2-105b-40b2-bb44-fa2251d17d2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:45.229978 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a895a2-105b-40b2-bb44-fa2251d17d2f admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:45.319075 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352037b-8d4a-4358-ac8c-4df3d2c58ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:45.321908 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352037b-8d4a-4358-ac8c-4df3d2c58ada admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:45.322405 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352037b-8d4a-4358-ac8c-4df3d2c58ada admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:45.323055 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0352037b-8d4a-4358-ac8c-4df3d2c58ada admin admin] PATH_INFO: `/v3/users/666692a600484d7db9f8e4b3b8427bab` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:45.325134 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0352037b-8d4a-4358-ac8c-4df3d2c58ada admin admin] RBAC: Authorizing `identity:delete_user(user_id=666692a600484d7db9f8e4b3b8427bab)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:45.327817 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0352037b-8d4a-4358-ac8c-4df3d2c58ada admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:45.392466 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-0352037b-8d4a-4358-ac8c-4df3d2c58ada admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '666692a600484d7db9f8e4b3b8427bab'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:45.408157 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-0352037b-8d4a-4358-ac8c-4df3d2c58ada admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '666692a600484d7db9f8e4b3b8427bab'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:45.423953 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-0352037b-8d4a-4358-ac8c-4df3d2c58ada admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '666692a600484d7db9f8e4b3b8427bab'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:45.433157 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-0352037b-8d4a-4358-ac8c-4df3d2c58ada admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '666692a600484d7db9f8e4b3b8427bab'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:45.434006 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-0352037b-8d4a-4358-ac8c-4df3d2c58ada admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '666692a600484d7db9f8e4b3b8427bab'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:45.442651 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 872/1742] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:06:45 2022] DELETE /identity/v3/users/666692a600484d7db9f8e4b3b8427bab => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:45.693447 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022acc2b-f361-4dc6-b3dc-fe1dbefa8b5d admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:45.696896 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-022acc2b-f361-4dc6-b3dc-fe1dbefa8b5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:45.744182 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022acc2b-f361-4dc6-b3dc-fe1dbefa8b5d admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:45.856704 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:45.904447 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a7b52-d918-4b1a-9ff9-3ec5608ab624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:45.907258 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352037b-8d4a-4358-ac8c-4df3d2c58ada admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:45.908757 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-0352037b-8d4a-4358-ac8c-4df3d2c58ada admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:45.911671 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a7b52-d918-4b1a-9ff9-3ec5608ab624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:45.912468 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a7b52-d918-4b1a-9ff9-3ec5608ab624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:45.912985 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a7b52-d918-4b1a-9ff9-3ec5608ab624 admin admin] PATH_INFO: `/v3/users/3388d85f2b3c477d8e392a3c6ce47f11` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:45.916733 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a7b52-d918-4b1a-9ff9-3ec5608ab624 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3388d85f2b3c477d8e392a3c6ce47f11)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:45.922865 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a7b52-d918-4b1a-9ff9-3ec5608ab624 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:45.932016 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0352037b-8d4a-4358-ac8c-4df3d2c58ada admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:45.940878 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464d4030-0735-4c81-92fc-8938d369e4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:45.941307 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d4030-0735-4c81-92fc-8938d369e4ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:45.941592 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d4030-0735-4c81-92fc-8938d369e4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:45.941870 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464d4030-0735-4c81-92fc-8938d369e4ba admin admin] PATH_INFO: `/v3/projects/b96e70dba40f475891ea2a40158cbf3e` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:45.943806 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d4030-0735-4c81-92fc-8938d369e4ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=b96e70dba40f475891ea2a40158cbf3e)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:45.947105 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464d4030-0735-4c81-92fc-8938d369e4ba admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:46.028141 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-0d1a7b52-d918-4b1a-9ff9-3ec5608ab624 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3388d85f2b3c477d8e392a3c6ce47f11'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:46.064078 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-0d1a7b52-d918-4b1a-9ff9-3ec5608ab624 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3388d85f2b3c477d8e392a3c6ce47f11'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:46.080234 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-0d1a7b52-d918-4b1a-9ff9-3ec5608ab624 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3388d85f2b3c477d8e392a3c6ce47f11'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:46.081483 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-0d1a7b52-d918-4b1a-9ff9-3ec5608ab624 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3388d85f2b3c477d8e392a3c6ce47f11'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:46.092404 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-464d4030-0735-4c81-92fc-8938d369e4ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b96e70dba40f475891ea2a40158cbf3e', 'request_id': 'req-464d4030-0735-4c81-92fc-8938d369e4ba'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:46.107530 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-0d1a7b52-d918-4b1a-9ff9-3ec5608ab624 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3388d85f2b3c477d8e392a3c6ce47f11'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:46.130107 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-464d4030-0735-4c81-92fc-8938d369e4ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b96e70dba40f475891ea2a40158cbf3e', 'request_id': 'req-464d4030-0735-4c81-92fc-8938d369e4ba'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:46.143742 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 871/1743] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:06:45 2022] DELETE /identity/v3/users/3388d85f2b3c477d8e392a3c6ce47f11 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:46.148157 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6e342-504f-49df-9e4d-9b56edd66b39 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:46.150309 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-464d4030-0735-4c81-92fc-8938d369e4ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b96e70dba40f475891ea2a40158cbf3e', 'request_id': 'req-464d4030-0735-4c81-92fc-8938d369e4ba'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:46.151242 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6e342-504f-49df-9e4d-9b56edd66b39 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:46.152042 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c6e342-504f-49df-9e4d-9b56edd66b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:46.186920 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:46.198981 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-464d4030-0735-4c81-92fc-8938d369e4ba admin admin] The token cache is being invalidate because project b96e70dba40f475891ea2a40158cbf3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:06:46.199497 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-464d4030-0735-4c81-92fc-8938d369e4ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:46.203826 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-464d4030-0735-4c81-92fc-8938d369e4ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:46.205945 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-00c6e342-504f-49df-9e4d-9b56edd66b39 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:46.206181 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 873/1744] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:06:45 2022] DELETE /identity/v3/projects/b96e70dba40f475891ea2a40158cbf3e => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:46.208352 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-00c6e342-504f-49df-9e4d-9b56edd66b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:46.211084 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9400d6-8ca4-4178-8677-8211a6f7332f None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:46.211517 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9400d6-8ca4-4178-8677-8211a6f7332f None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:46.212021 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9400d6-8ca4-4178-8677-8211a6f7332f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:46.256594 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-bb9400d6-8ca4-4178-8677-8211a6f7332f None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:46.258516 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-bb9400d6-8ca4-4178-8677-8211a6f7332f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:46.329281 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 872/1745] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:46.341334 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6e342-504f-49df-9e4d-9b56edd66b39 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:46.344178 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-00c6e342-504f-49df-9e4d-9b56edd66b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:46.355203 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 874/1746] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:46.364038 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8bfb2-cd3d-421c-b4ca-3738c516b3de None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:46.364642 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8bfb2-cd3d-421c-b4ca-3738c516b3de None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:46.365886 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a8bfb2-cd3d-421c-b4ca-3738c516b3de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:46.400941 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-83a8bfb2-cd3d-421c-b4ca-3738c516b3de None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:46.403608 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-83a8bfb2-cd3d-421c-b4ca-3738c516b3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:46.424276 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 875/1747] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:46.429425 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8bfb2-cd3d-421c-b4ca-3738c516b3de None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:46.440131 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a8bfb2-cd3d-421c-b4ca-3738c516b3de None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:46.446322 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb4690e-0597-4174-b92c-723e78f3482e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:46.447202 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4690e-0597-4174-b92c-723e78f3482e admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:46.447698 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4690e-0597-4174-b92c-723e78f3482e admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:46.448360 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb4690e-0597-4174-b92c-723e78f3482e admin admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:46.450000 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb4690e-0597-4174-b92c-723e78f3482e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:06:46.450549 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb4690e-0597-4174-b92c-723e78f3482e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:46.452496 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb4690e-0597-4174-b92c-723e78f3482e admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:46.475044 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 876/1748] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:06:46.480378 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb4690e-0597-4174-b92c-723e78f3482e admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:46.492006 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb4690e-0597-4174-b92c-723e78f3482e admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:46.496880 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c6e342-504f-49df-9e4d-9b56edd66b39 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:46.498654 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92565183-76b4-43aa-a345-503634f39764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:46.499286 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92565183-76b4-43aa-a345-503634f39764 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:46.499573 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92565183-76b4-43aa-a345-503634f39764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:46.499911 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92565183-76b4-43aa-a345-503634f39764 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:46.501598 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92565183-76b4-43aa-a345-503634f39764 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:46.504789 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92565183-76b4-43aa-a345-503634f39764 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:46.509194 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e1c94-0064-4c76-bb67-286eea55ee36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:46.509849 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e1c94-0064-4c76-bb67-286eea55ee36 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:46.510135 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e1c94-0064-4c76-bb67-286eea55ee36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:46.510452 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339e1c94-0064-4c76-bb67-286eea55ee36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:46.537429 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339e1c94-0064-4c76-bb67-286eea55ee36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:46.539717 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339e1c94-0064-4c76-bb67-286eea55ee36 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:46.550862 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 877/1749] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:06:46 2022] POST /identity/v3/projects => generated 385 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:06:46.554189 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92565183-76b4-43aa-a345-503634f39764 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:46.567890 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 873/1750] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:06:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:06:46.658861 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92565183-76b4-43aa-a345-503634f39764 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:46.665422 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b6a210-423f-4e5c-93b0-47b958804fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:46.667784 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6a210-423f-4e5c-93b0-47b958804fed admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:46.667784 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6a210-423f-4e5c-93b0-47b958804fed admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:46.667784 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b6a210-423f-4e5c-93b0-47b958804fed admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:46.682873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b6a210-423f-4e5c-93b0-47b958804fed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:46.701184 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b6a210-423f-4e5c-93b0-47b958804fed admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:46.742697 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 878/1751] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:06:46 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:06:46.747518 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e1c94-0064-4c76-bb67-286eea55ee36 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:46.753870 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:46.764183 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339e1c94-0064-4c76-bb67-286eea55ee36 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:46.773187 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa794595-9920-4628-9fe2-e8738810b82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:46.774433 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa794595-9920-4628-9fe2-e8738810b82d admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:46.775432 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa794595-9920-4628-9fe2-e8738810b82d admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:46.775978 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa794595-9920-4628-9fe2-e8738810b82d admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:46.777360 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa794595-9920-4628-9fe2-e8738810b82d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:46.780788 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa794595-9920-4628-9fe2-e8738810b82d admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:46.801006 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 874/1752] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:06:46.802108 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b6a210-423f-4e5c-93b0-47b958804fed admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:46.823206 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b6a210-423f-4e5c-93b0-47b958804fed admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:46.830220 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cefbdf8-b10c-41e5-ad18-3c1368e1bdb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:46.831071 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefbdf8-b10c-41e5-ad18-3c1368e1bdb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:46.831828 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefbdf8-b10c-41e5-ad18-3c1368e1bdb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:46.832210 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefbdf8-b10c-41e5-ad18-3c1368e1bdb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:46.841645 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cefbdf8-b10c-41e5-ad18-3c1368e1bdb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:46.845460 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cefbdf8-b10c-41e5-ad18-3c1368e1bdb2 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:46.870158 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa794595-9920-4628-9fe2-e8738810b82d admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:46.899143 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 879/1753] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:06:46.941648 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa794595-9920-4628-9fe2-e8738810b82d admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:46.971824 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e1fd1-3027-4e00-b24d-bbdd8a4008b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:46.975654 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e1fd1-3027-4e00-b24d-bbdd8a4008b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:46.976020 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e1fd1-3027-4e00-b24d-bbdd8a4008b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:46.976373 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73e1fd1-3027-4e00-b24d-bbdd8a4008b7 admin admin] PATH_INFO: `/v3/projects/8d61c457bb9e4d259c25d48cb80761b6/users/41a400ff55054e70858c51cfcce3a1c2/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:47.050556 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73e1fd1-3027-4e00-b24d-bbdd8a4008b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d61c457bb9e4d259c25d48cb80761b6, user_id=41a400ff55054e70858c51cfcce3a1c2, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:47.057959 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73e1fd1-3027-4e00-b24d-bbdd8a4008b7 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:47.127524 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 875/1754] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:06:46 2022] PUT /identity/v3/projects/8d61c457bb9e4d259c25d48cb80761b6/users/41a400ff55054e70858c51cfcce3a1c2/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:47.138939 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d212795-4c1d-4dd1-95bc-0af2af57cf8f None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:47.139594 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d212795-4c1d-4dd1-95bc-0af2af57cf8f None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:47.140077 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d212795-4c1d-4dd1-95bc-0af2af57cf8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:47.290612 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-0d212795-4c1d-4dd1-95bc-0af2af57cf8f None None] MFA Rules not processed for user `41a400ff55054e70858c51cfcce3a1c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:47.292444 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-0d212795-4c1d-4dd1-95bc-0af2af57cf8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:47.414236 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:47.558291 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 880/1755] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 420 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:47.561919 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e1fd1-3027-4e00-b24d-bbdd8a4008b7 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:47.689803 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d212795-4c1d-4dd1-95bc-0af2af57cf8f None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:47.691504 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-0d212795-4c1d-4dd1-95bc-0af2af57cf8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:47.762608 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73e1fd1-3027-4e00-b24d-bbdd8a4008b7 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:47.768461 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd4b61-4f1b-4a7f-bb9b-7bd25e062225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:47.770633 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd4b61-4f1b-4a7f-bb9b-7bd25e062225 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:47.771206 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd4b61-4f1b-4a7f-bb9b-7bd25e062225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:47.772830 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90dd4b61-4f1b-4a7f-bb9b-7bd25e062225 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:47.773720 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd4b61-4f1b-4a7f-bb9b-7bd25e062225 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:47.775935 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90dd4b61-4f1b-4a7f-bb9b-7bd25e062225 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:47.862647 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 876/1756] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:06:47 2022] POST /identity/v3/projects => generated 385 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:06:47.864407 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd4b61-4f1b-4a7f-bb9b-7bd25e062225 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:47.866902 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-90dd4b61-4f1b-4a7f-bb9b-7bd25e062225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:47.937192 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d212795-4c1d-4dd1-95bc-0af2af57cf8f None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:48.055698 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90dd4b61-4f1b-4a7f-bb9b-7bd25e062225 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:48.061850 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b0246-96e1-4ebf-83e6-873848185809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:48.063158 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a116b3-1fd5-4fe8-ac06-32d196c1a217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:48.064097 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a116b3-1fd5-4fe8-ac06-32d196c1a217 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:48.064573 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a116b3-1fd5-4fe8-ac06-32d196c1a217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:48.065037 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a116b3-1fd5-4fe8-ac06-32d196c1a217 admin admin] PATH_INFO: `/v3/projects/3660c6fffa274060ada1984b56df5578` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:48.066130 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b0246-96e1-4ebf-83e6-873848185809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:48.066481 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b0246-96e1-4ebf-83e6-873848185809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:48.066819 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b0246-96e1-4ebf-83e6-873848185809 admin admin] PATH_INFO: `/v3/users/dc82bd89cd4d407a85fe59a5fee5f2b0` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:48.067375 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a116b3-1fd5-4fe8-ac06-32d196c1a217 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3660c6fffa274060ada1984b56df5578)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:48.070427 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a116b3-1fd5-4fe8-ac06-32d196c1a217 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:48.071674 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b0246-96e1-4ebf-83e6-873848185809 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc82bd89cd4d407a85fe59a5fee5f2b0)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:48.079142 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65b0246-96e1-4ebf-83e6-873848185809 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:48.190099 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-61a116b3-1fd5-4fe8-ac06-32d196c1a217 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3660c6fffa274060ada1984b56df5578', 'request_id': 'req-61a116b3-1fd5-4fe8-ac06-32d196c1a217'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:48.198809 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:48.205638 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-61a116b3-1fd5-4fe8-ac06-32d196c1a217 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3660c6fffa274060ada1984b56df5578', 'request_id': 'req-61a116b3-1fd5-4fe8-ac06-32d196c1a217'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:48.214640 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c65b0246-96e1-4ebf-83e6-873848185809 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc82bd89cd4d407a85fe59a5fee5f2b0'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:48.222334 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-61a116b3-1fd5-4fe8-ac06-32d196c1a217 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3660c6fffa274060ada1984b56df5578', 'request_id': 'req-61a116b3-1fd5-4fe8-ac06-32d196c1a217'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:48.246298 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c65b0246-96e1-4ebf-83e6-873848185809 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc82bd89cd4d407a85fe59a5fee5f2b0'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:48.247470 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-61a116b3-1fd5-4fe8-ac06-32d196c1a217 admin admin] The token cache is being invalidate because project 3660c6fffa274060ada1984b56df5578 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:06:48.247757 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-61a116b3-1fd5-4fe8-ac06-32d196c1a217 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:48.250310 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-61a116b3-1fd5-4fe8-ac06-32d196c1a217 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:48.255274 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 877/1757] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:06:47 2022] DELETE /identity/v3/projects/3660c6fffa274060ada1984b56df5578 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:48.255274 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a116b3-1fd5-4fe8-ac06-32d196c1a217 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:48.256569 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-61a116b3-1fd5-4fe8-ac06-32d196c1a217 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:48.262426 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c65b0246-96e1-4ebf-83e6-873848185809 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc82bd89cd4d407a85fe59a5fee5f2b0'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:48.265922 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:48.281285 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c65b0246-96e1-4ebf-83e6-873848185809 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc82bd89cd4d407a85fe59a5fee5f2b0'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:48.282527 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c65b0246-96e1-4ebf-83e6-873848185809 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc82bd89cd4d407a85fe59a5fee5f2b0'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:48.292566 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 881/1758] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:06:47 2022] DELETE /identity/v3/users/dc82bd89cd4d407a85fe59a5fee5f2b0 => generated 0 bytes in 604 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:48.296757 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b0246-96e1-4ebf-83e6-873848185809 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:48.298193 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-c65b0246-96e1-4ebf-83e6-873848185809 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:48.398685 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a116b3-1fd5-4fe8-ac06-32d196c1a217 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:48.440317 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b0246-96e1-4ebf-83e6-873848185809 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:48.455857 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff43e1-3f7e-4936-8f78-ab84387f89f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:48.456767 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff43e1-3f7e-4936-8f78-ab84387f89f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:48.457206 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff43e1-3f7e-4936-8f78-ab84387f89f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:48.457642 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff43e1-3f7e-4936-8f78-ab84387f89f5 admin admin] PATH_INFO: `/v3/users/a412e296ea9c4753adf440b9fc5e356a` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:48.459173 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ff43e1-3f7e-4936-8f78-ab84387f89f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a412e296ea9c4753adf440b9fc5e356a)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:48.461714 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ff43e1-3f7e-4936-8f78-ab84387f89f5 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:48.554237 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8f5cbc-eaa2-4172-8741-720997232afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:48.554965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f5cbc-eaa2-4172-8741-720997232afe admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:48.555246 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f5cbc-eaa2-4172-8741-720997232afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:48.555519 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8f5cbc-eaa2-4172-8741-720997232afe admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:48.556516 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8f5cbc-eaa2-4172-8741-720997232afe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:48.558593 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8f5cbc-eaa2-4172-8741-720997232afe admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:48.586048 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-22ff43e1-3f7e-4936-8f78-ab84387f89f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a412e296ea9c4753adf440b9fc5e356a'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:48.605059 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-22ff43e1-3f7e-4936-8f78-ab84387f89f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a412e296ea9c4753adf440b9fc5e356a'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:48.614412 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-22ff43e1-3f7e-4936-8f78-ab84387f89f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a412e296ea9c4753adf440b9fc5e356a'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:48.627745 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 878/1759] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:06:48 2022] POST /identity/v3/users => generated 293 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:06:48.632496 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf0dde-9746-48fd-920b-5cd8a2525f21 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:48.632775 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf0dde-9746-48fd-920b-5cd8a2525f21 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:48.633061 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf0dde-9746-48fd-920b-5cd8a2525f21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:48.634007 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-22ff43e1-3f7e-4936-8f78-ab84387f89f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a412e296ea9c4753adf440b9fc5e356a'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:48.634692 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-22ff43e1-3f7e-4936-8f78-ab84387f89f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a412e296ea9c4753adf440b9fc5e356a'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:48.645042 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 882/1760] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:06:48 2022] DELETE /identity/v3/users/a412e296ea9c4753adf440b9fc5e356a => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:48.646801 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff43e1-3f7e-4936-8f78-ab84387f89f5 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:48.648882 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-22ff43e1-3f7e-4936-8f78-ab84387f89f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:48.682429 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-4ebf0dde-9746-48fd-920b-5cd8a2525f21 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:48.683668 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4ebf0dde-9746-48fd-920b-5cd8a2525f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:48.809089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff43e1-3f7e-4936-8f78-ab84387f89f5 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:48.819662 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ffb0fd-917a-48d5-9270-dba69e822c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:48.820300 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffb0fd-917a-48d5-9270-dba69e822c26 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:48.821501 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffb0fd-917a-48d5-9270-dba69e822c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:48.821777 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffb0fd-917a-48d5-9270-dba69e822c26 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:48.822469 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ffb0fd-917a-48d5-9270-dba69e822c26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:48.827159 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ffb0fd-917a-48d5-9270-dba69e822c26 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:48.837179 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 879/1761] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:48.850950 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf0dde-9746-48fd-920b-5cd8a2525f21 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:48.854816 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4ebf0dde-9746-48fd-920b-5cd8a2525f21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:48.863425 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 883/1762] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:06:48.872638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ffb0fd-917a-48d5-9270-dba69e822c26 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:48.897152 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ffb0fd-917a-48d5-9270-dba69e822c26 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:48.913116 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32a8a4-3940-4e91-b967-24580726b4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:48.913747 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32a8a4-3940-4e91-b967-24580726b4ea admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:48.916827 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32a8a4-3940-4e91-b967-24580726b4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:48.917111 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e32a8a4-3940-4e91-b967-24580726b4ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:48.919044 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:48.919375 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e32a8a4-3940-4e91-b967-24580726b4ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:48.921014 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e32a8a4-3940-4e91-b967-24580726b4ea admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:48.943743 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:48.981343 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 884/1763] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:06:48.985046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32a8a4-3940-4e91-b967-24580726b4ea admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:49.016817 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e32a8a4-3940-4e91-b967-24580726b4ea admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:49.040675 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa679fe7-0de0-4aae-a12c-431facdb98fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:49.041359 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa679fe7-0de0-4aae-a12c-431facdb98fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:49.042018 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa679fe7-0de0-4aae-a12c-431facdb98fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:49.042018 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa679fe7-0de0-4aae-a12c-431facdb98fb admin admin] PATH_INFO: `/v3/projects/7917c227a1554aae95b1c26a3f61309a/users/071b60bea6ef41bfb58991f8f5d86c51/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:49.077871 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf0dde-9746-48fd-920b-5cd8a2525f21 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:49.084728 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a50dd-04dd-4de7-8641-53d3b042893f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:49.085363 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a50dd-04dd-4de7-8641-53d3b042893f service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:49.085632 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a50dd-04dd-4de7-8641-53d3b042893f service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:49.085906 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98a50dd-04dd-4de7-8641-53d3b042893f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:49.093204 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa679fe7-0de0-4aae-a12c-431facdb98fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7917c227a1554aae95b1c26a3f61309a, user_id=071b60bea6ef41bfb58991f8f5d86c51, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:49.095974 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa679fe7-0de0-4aae-a12c-431facdb98fb admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:49.122640 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a50dd-04dd-4de7-8641-53d3b042893f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:49.124157 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98a50dd-04dd-4de7-8641-53d3b042893f service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:49.138359 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 885/1764] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:06:48 2022] PUT /identity/v3/projects/7917c227a1554aae95b1c26a3f61309a/users/071b60bea6ef41bfb58991f8f5d86c51/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:49.148122 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be400582-a76f-4aee-924e-6a3253716c83 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:49.148635 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be400582-a76f-4aee-924e-6a3253716c83 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:49.149060 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be400582-a76f-4aee-924e-6a3253716c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:49.173489 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 880/1765] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:06:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:06:49.211239 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-be400582-a76f-4aee-924e-6a3253716c83 None None] MFA Rules not processed for user `071b60bea6ef41bfb58991f8f5d86c51`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:49.212996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-be400582-a76f-4aee-924e-6a3253716c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:49.360509 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 886/1766] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:49.363971 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a50dd-04dd-4de7-8641-53d3b042893f service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:49.380673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98a50dd-04dd-4de7-8641-53d3b042893f service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:49.517439 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c244ee4-6de4-494d-8c37-c40419a1fa76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:49.518008 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c244ee4-6de4-494d-8c37-c40419a1fa76 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:49.518320 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c244ee4-6de4-494d-8c37-c40419a1fa76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:49.518651 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c244ee4-6de4-494d-8c37-c40419a1fa76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:49.519245 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c244ee4-6de4-494d-8c37-c40419a1fa76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:49.521725 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c244ee4-6de4-494d-8c37-c40419a1fa76 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:49.593401 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 881/1767] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:06:49 2022] POST /identity/v3/projects => generated 381 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:06:49.597735 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be400582-a76f-4aee-924e-6a3253716c83 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:49.628172 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be400582-a76f-4aee-924e-6a3253716c83 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:49.799797 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935de396-1639-4bbe-8fe9-b5ce2684ccb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:49.800733 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935de396-1639-4bbe-8fe9-b5ce2684ccb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:49.801097 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935de396-1639-4bbe-8fe9-b5ce2684ccb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:49.801439 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935de396-1639-4bbe-8fe9-b5ce2684ccb4 admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:49.802491 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935de396-1639-4bbe-8fe9-b5ce2684ccb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:49.805979 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935de396-1639-4bbe-8fe9-b5ce2684ccb4 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:49.855671 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 887/1768] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:06:49 2022] POST /identity/v3/users => generated 297 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:06:49.860245 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c244ee4-6de4-494d-8c37-c40419a1fa76 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:49.878321 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c244ee4-6de4-494d-8c37-c40419a1fa76 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:49.891901 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea5845-eea0-4e9c-8222-7661520456e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:49.892958 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea5845-eea0-4e9c-8222-7661520456e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:49.893492 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea5845-eea0-4e9c-8222-7661520456e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:49.893911 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ea5845-eea0-4e9c-8222-7661520456e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:49.894748 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea5845-eea0-4e9c-8222-7661520456e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:49.900138 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ea5845-eea0-4e9c-8222-7661520456e9 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:49.929366 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 882/1769] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:06:49.933325 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935de396-1639-4bbe-8fe9-b5ce2684ccb4 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:49.949732 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935de396-1639-4bbe-8fe9-b5ce2684ccb4 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:49.961124 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55923e79-8952-404e-9e64-e924d204ea0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:49.961834 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55923e79-8952-404e-9e64-e924d204ea0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:49.962116 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55923e79-8952-404e-9e64-e924d204ea0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:49.962421 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55923e79-8952-404e-9e64-e924d204ea0c admin admin] PATH_INFO: `/v3/projects/178f0ba912e940ceb156af21b615d9d3/users/123c7ca57afb4186a8cdb9c4f890b08a/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:49.998039 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55923e79-8952-404e-9e64-e924d204ea0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=178f0ba912e940ceb156af21b615d9d3, user_id=123c7ca57afb4186a8cdb9c4f890b08a, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:50.001295 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55923e79-8952-404e-9e64-e924d204ea0c admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:50.036017 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 888/1770] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:06:49 2022] PUT /identity/v3/projects/178f0ba912e940ceb156af21b615d9d3/users/123c7ca57afb4186a8cdb9c4f890b08a/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:50.040456 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea5845-eea0-4e9c-8222-7661520456e9 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:50.060590 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ea5845-eea0-4e9c-8222-7661520456e9 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:50.171828 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55923e79-8952-404e-9e64-e924d204ea0c admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:50.173898 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-55923e79-8952-404e-9e64-e924d204ea0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:50.261221 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af3c8ff-0c75-47d2-9af8-2766e788509c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:50.262223 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:50.262614 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af3c8ff-0c75-47d2-9af8-2766e788509c admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:50.262907 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af3c8ff-0c75-47d2-9af8-2766e788509c admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:50.263489 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af3c8ff-0c75-47d2-9af8-2766e788509c admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:50.264917 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af3c8ff-0c75-47d2-9af8-2766e788509c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:50.268786 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af3c8ff-0c75-47d2-9af8-2766e788509c admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:50.295073 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 883/1771] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:50 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:06:50.300613 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af3c8ff-0c75-47d2-9af8-2766e788509c admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:50.330817 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af3c8ff-0c75-47d2-9af8-2766e788509c admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:50.345405 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f097ca7c-d25a-463a-8a7c-0a7b7147ba01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:50.346277 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097ca7c-d25a-463a-8a7c-0a7b7147ba01 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:50.346623 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097ca7c-d25a-463a-8a7c-0a7b7147ba01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:50.346970 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f097ca7c-d25a-463a-8a7c-0a7b7147ba01 admin admin] PATH_INFO: `/v3/projects/178f0ba912e940ceb156af21b615d9d3/users/123c7ca57afb4186a8cdb9c4f890b08a/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:50.353137 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f097ca7c-d25a-463a-8a7c-0a7b7147ba01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=178f0ba912e940ceb156af21b615d9d3, user_id=123c7ca57afb4186a8cdb9c4f890b08a, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:50.354817 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55923e79-8952-404e-9e64-e924d204ea0c admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:50.358067 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f097ca7c-d25a-463a-8a7c-0a7b7147ba01 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:50.392554 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1154c8d-33c8-4fe0-a9f3-9412e9cea38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:50.393170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1154c8d-33c8-4fe0-a9f3-9412e9cea38f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:50.393464 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1154c8d-33c8-4fe0-a9f3-9412e9cea38f admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:50.393815 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1154c8d-33c8-4fe0-a9f3-9412e9cea38f admin admin] PATH_INFO: `/v3/projects/4182ccd297e54d4b8fad97db74777b7c` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:50.398099 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 884/1772] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:06:50 2022] PUT /identity/v3/projects/178f0ba912e940ceb156af21b615d9d3/users/123c7ca57afb4186a8cdb9c4f890b08a/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:50.399191 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f097ca7c-d25a-463a-8a7c-0a7b7147ba01 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:50.399672 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1154c8d-33c8-4fe0-a9f3-9412e9cea38f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4182ccd297e54d4b8fad97db74777b7c)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:50.403801 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1154c8d-33c8-4fe0-a9f3-9412e9cea38f admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:50.414004 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f097ca7c-d25a-463a-8a7c-0a7b7147ba01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:50.467574 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:50.495415 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c1154c8d-33c8-4fe0-a9f3-9412e9cea38f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4182ccd297e54d4b8fad97db74777b7c', 'request_id': 'req-c1154c8d-33c8-4fe0-a9f3-9412e9cea38f'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:50.517884 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c1154c8d-33c8-4fe0-a9f3-9412e9cea38f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4182ccd297e54d4b8fad97db74777b7c', 'request_id': 'req-c1154c8d-33c8-4fe0-a9f3-9412e9cea38f'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:50.535038 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c1154c8d-33c8-4fe0-a9f3-9412e9cea38f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4182ccd297e54d4b8fad97db74777b7c', 'request_id': 'req-c1154c8d-33c8-4fe0-a9f3-9412e9cea38f'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:50.553991 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c1154c8d-33c8-4fe0-a9f3-9412e9cea38f admin admin] The token cache is being invalidate because project 4182ccd297e54d4b8fad97db74777b7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:06:50.554661 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c1154c8d-33c8-4fe0-a9f3-9412e9cea38f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:50.555603 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c1154c8d-33c8-4fe0-a9f3-9412e9cea38f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:50.566640 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 889/1773] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:06:50 2022] DELETE /identity/v3/projects/4182ccd297e54d4b8fad97db74777b7c => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:50.568267 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f097ca7c-d25a-463a-8a7c-0a7b7147ba01 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:50.568787 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3866f-0c71-46c3-97ff-a6b0fe22b20e None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:50.569520 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3866f-0c71-46c3-97ff-a6b0fe22b20e None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:50.569996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b3866f-0c71-46c3-97ff-a6b0fe22b20e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:50.608170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-99b3866f-0c71-46c3-97ff-a6b0fe22b20e None None] MFA Rules not processed for user `123c7ca57afb4186a8cdb9c4f890b08a`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:50.610177 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-99b3866f-0c71-46c3-97ff-a6b0fe22b20e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:50.632530 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ccc43-7e23-4c1a-9fe4-39469854ab28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:50.633043 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ccc43-7e23-4c1a-9fe4-39469854ab28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:50.633400 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ccc43-7e23-4c1a-9fe4-39469854ab28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:50.633746 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017ccc43-7e23-4c1a-9fe4-39469854ab28 admin admin] PATH_INFO: `/v3/projects/59bf64a6275a44599b7772f80c482626` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:50.635556 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017ccc43-7e23-4c1a-9fe4-39469854ab28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59bf64a6275a44599b7772f80c482626)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:50.637015 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017ccc43-7e23-4c1a-9fe4-39469854ab28 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:50.698389 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-017ccc43-7e23-4c1a-9fe4-39469854ab28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59bf64a6275a44599b7772f80c482626', 'request_id': 'req-017ccc43-7e23-4c1a-9fe4-39469854ab28'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:50.710378 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-017ccc43-7e23-4c1a-9fe4-39469854ab28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59bf64a6275a44599b7772f80c482626', 'request_id': 'req-017ccc43-7e23-4c1a-9fe4-39469854ab28'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:50.711160 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 890/1774] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:50 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:50.713182 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581296db-a30a-41a3-bf3a-4354c2b2cec9 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:50.713734 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581296db-a30a-41a3-bf3a-4354c2b2cec9 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:50.714209 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581296db-a30a-41a3-bf3a-4354c2b2cec9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:50.727718 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-017ccc43-7e23-4c1a-9fe4-39469854ab28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59bf64a6275a44599b7772f80c482626', 'request_id': 'req-017ccc43-7e23-4c1a-9fe4-39469854ab28'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:50.744674 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-017ccc43-7e23-4c1a-9fe4-39469854ab28 admin admin] The token cache is being invalidate because project 59bf64a6275a44599b7772f80c482626 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:06:50.745053 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-017ccc43-7e23-4c1a-9fe4-39469854ab28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:50.745778 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-017ccc43-7e23-4c1a-9fe4-39469854ab28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:50.747459 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-581296db-a30a-41a3-bf3a-4354c2b2cec9 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:50.748251 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 885/1775] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:06:50 2022] DELETE /identity/v3/projects/59bf64a6275a44599b7772f80c482626 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:50.749131 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-581296db-a30a-41a3-bf3a-4354c2b2cec9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:50.749250 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ccc43-7e23-4c1a-9fe4-39469854ab28 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:50.750825 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-017ccc43-7e23-4c1a-9fe4-39469854ab28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:50.804738 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:50.847367 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017ccc43-7e23-4c1a-9fe4-39469854ab28 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:50.851727 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37460e2d-6512-4f0b-b0da-a9fdfbb5479e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:50.852379 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37460e2d-6512-4f0b-b0da-a9fdfbb5479e service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:50.852659 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37460e2d-6512-4f0b-b0da-a9fdfbb5479e service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:50.852930 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37460e2d-6512-4f0b-b0da-a9fdfbb5479e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:50.854433 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 891/1776] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:50.854963 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-37460e2d-6512-4f0b-b0da-a9fdfbb5479e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:50.856473 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d42d7f-fd89-4589-aeef-356117c96a45 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:50.856935 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d42d7f-fd89-4589-aeef-356117c96a45 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:50.857213 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d42d7f-fd89-4589-aeef-356117c96a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:50.870378 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:50.887803 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-54d42d7f-fd89-4589-aeef-356117c96a45 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:50.889062 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-54d42d7f-fd89-4589-aeef-356117c96a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:50.901281 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 892/1777] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:50.911038 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50dcae6-6973-4f49-a797-e2e84e8d7d73 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:50.911510 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50dcae6-6973-4f49-a797-e2e84e8d7d73 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:50.912008 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50dcae6-6973-4f49-a797-e2e84e8d7d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:50.937008 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-c50dcae6-6973-4f49-a797-e2e84e8d7d73 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:50.939817 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-c50dcae6-6973-4f49-a797-e2e84e8d7d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:50.941883 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37460e2d-6512-4f0b-b0da-a9fdfbb5479e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:50.943439 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37460e2d-6512-4f0b-b0da-a9fdfbb5479e service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:50.951367 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 893/1778] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:50.953536 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4bb4f8-c69a-4fcb-ad53-c1bf1b242485 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:50.953680 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4bb4f8-c69a-4fcb-ad53-c1bf1b242485 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:50.953873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4bb4f8-c69a-4fcb-ad53-c1bf1b242485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:50.977722 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-3f4bb4f8-c69a-4fcb-ad53-c1bf1b242485 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:50.979018 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-3f4bb4f8-c69a-4fcb-ad53-c1bf1b242485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:50.983317 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 886/1779] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:06:50 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:06:50.986163 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37460e2d-6512-4f0b-b0da-a9fdfbb5479e service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:50.992852 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 894/1780] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:50.997075 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4bb4f8-c69a-4fcb-ad53-c1bf1b242485 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:51.012097 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4bb4f8-c69a-4fcb-ad53-c1bf1b242485 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:51.018869 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00602f-0c2e-4bc7-a8e9-2d84433f5079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:51.019753 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00602f-0c2e-4bc7-a8e9-2d84433f5079 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:51.020179 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00602f-0c2e-4bc7-a8e9-2d84433f5079 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:51.020594 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00602f-0c2e-4bc7-a8e9-2d84433f5079 admin admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:51.022388 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab00602f-0c2e-4bc7-a8e9-2d84433f5079 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:06:51.022670 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37460e2d-6512-4f0b-b0da-a9fdfbb5479e service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:51.023013 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab00602f-0c2e-4bc7-a8e9-2d84433f5079 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:51.025018 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab00602f-0c2e-4bc7-a8e9-2d84433f5079 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:51.032485 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb8275-3a8e-4049-a343-465183c23c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:51.033064 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb8275-3a8e-4049-a343-465183c23c69 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:51.035678 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb8275-3a8e-4049-a343-465183c23c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:51.035970 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fb8275-3a8e-4049-a343-465183c23c69 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:51.036718 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fb8275-3a8e-4049-a343-465183c23c69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:06:51.037035 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fb8275-3a8e-4049-a343-465183c23c69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:51.039393 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 895/1781] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:06:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:06:51.040613 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fb8275-3a8e-4049-a343-465183c23c69 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:51.043705 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00602f-0c2e-4bc7-a8e9-2d84433f5079 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:51.056285 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00602f-0c2e-4bc7-a8e9-2d84433f5079 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:51.061428 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81c2a0e-2009-4eea-b114-add05f93874f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:51.061976 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c2a0e-2009-4eea-b114-add05f93874f admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:51.062276 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c2a0e-2009-4eea-b114-add05f93874f admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:51.062675 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81c2a0e-2009-4eea-b114-add05f93874f admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:51.063736 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81c2a0e-2009-4eea-b114-add05f93874f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:51.065797 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81c2a0e-2009-4eea-b114-add05f93874f admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:51.069082 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 887/1782] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:06:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:06:51.073584 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb8275-3a8e-4049-a343-465183c23c69 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:51.089494 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fb8275-3a8e-4049-a343-465183c23c69 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:51.099875 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d6219-9d69-4839-914c-c1e316a1dcda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:51.100674 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d6219-9d69-4839-914c-c1e316a1dcda admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:51.101062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d6219-9d69-4839-914c-c1e316a1dcda admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:51.101449 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d6219-9d69-4839-914c-c1e316a1dcda admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:51.102277 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d6219-9d69-4839-914c-c1e316a1dcda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:51.105707 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d6219-9d69-4839-914c-c1e316a1dcda admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:51.166785 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 896/1783] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:06:51 2022] POST /identity/v3/projects => generated 389 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:06:51.167816 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81c2a0e-2009-4eea-b114-add05f93874f admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:51.191516 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 888/1784] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:06:51 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:06:51.195580 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d6219-9d69-4839-914c-c1e316a1dcda admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:51.340407 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81c2a0e-2009-4eea-b114-add05f93874f admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:51.390587 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d6219-9d69-4839-914c-c1e316a1dcda admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:51.410876 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3fd145-2d71-4a91-86b1-fdacc55c0c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:51.412610 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fd145-2d71-4a91-86b1-fdacc55c0c30 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:51.412610 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fd145-2d71-4a91-86b1-fdacc55c0c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:51.414187 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3fd145-2d71-4a91-86b1-fdacc55c0c30 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:51.415108 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3fd145-2d71-4a91-86b1-fdacc55c0c30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:51.418531 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3fd145-2d71-4a91-86b1-fdacc55c0c30 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:51.475697 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 889/1785] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:06:51 2022] POST /identity/v3/users => generated 287 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:06:51.477698 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3fd145-2d71-4a91-86b1-fdacc55c0c30 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:51.496306 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3fd145-2d71-4a91-86b1-fdacc55c0c30 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:51.518833 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6a6c7-0db2-4a7c-90f7-9dc57da149e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:51.519469 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6a6c7-0db2-4a7c-90f7-9dc57da149e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:51.519796 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6a6c7-0db2-4a7c-90f7-9dc57da149e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:51.520018 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6a6c7-0db2-4a7c-90f7-9dc57da149e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:51.520669 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6a6c7-0db2-4a7c-90f7-9dc57da149e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:51.522480 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e6a6c7-0db2-4a7c-90f7-9dc57da149e5 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:51.528866 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c70141-b791-4a20-9238-45563c57dfb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:51.529539 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c70141-b791-4a20-9238-45563c57dfb7 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:51.529816 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c70141-b791-4a20-9238-45563c57dfb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:51.530095 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c70141-b791-4a20-9238-45563c57dfb7 admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:51.537462 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 890/1786] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:06:51.537684 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c70141-b791-4a20-9238-45563c57dfb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:51.538597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6a6c7-0db2-4a7c-90f7-9dc57da149e5 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:51.546829 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c70141-b791-4a20-9238-45563c57dfb7 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:51.553253 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e6a6c7-0db2-4a7c-90f7-9dc57da149e5 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:51.565091 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb6836c-f261-4859-a063-ecd28a90c930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:51.565818 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6836c-f261-4859-a063-ecd28a90c930 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:51.566111 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6836c-f261-4859-a063-ecd28a90c930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:51.566415 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb6836c-f261-4859-a063-ecd28a90c930 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:51.567725 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb6836c-f261-4859-a063-ecd28a90c930 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:51.571128 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb6836c-f261-4859-a063-ecd28a90c930 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:51.592243 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 897/1787] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:06:51 2022] POST /identity/v3/users => generated 295 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:06:51.600028 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c70141-b791-4a20-9238-45563c57dfb7 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:51.615868 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 891/1788] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 20:06:51.616991 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb6836c-f261-4859-a063-ecd28a90c930 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:51.618200 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c70141-b791-4a20-9238-45563c57dfb7 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:51.636115 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb6836c-f261-4859-a063-ecd28a90c930 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:51.637402 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f690ae-4835-4c40-b520-e14c4a393292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:51.638230 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f690ae-4835-4c40-b520-e14c4a393292 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:51.638367 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f690ae-4835-4c40-b520-e14c4a393292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:51.638623 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f690ae-4835-4c40-b520-e14c4a393292 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:51.639284 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f690ae-4835-4c40-b520-e14c4a393292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:51.640906 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f690ae-4835-4c40-b520-e14c4a393292 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:51.651283 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3456d7be-6b1b-487b-a1ba-88ee40bb727b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:51.651968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456d7be-6b1b-487b-a1ba-88ee40bb727b admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:51.652367 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456d7be-6b1b-487b-a1ba-88ee40bb727b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:51.652498 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3456d7be-6b1b-487b-a1ba-88ee40bb727b admin admin] PATH_INFO: `/v3/projects/46b77aad0457485693b9a43b85506915/users/48be37e3311a4d6c968b135f8e5ee536/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:51.664686 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 898/1789] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:06:51.669198 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f690ae-4835-4c40-b520-e14c4a393292 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:51.691864 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3456d7be-6b1b-487b-a1ba-88ee40bb727b admin admin] RBAC: Authorizing `identity:create_grant(project_id=46b77aad0457485693b9a43b85506915, user_id=48be37e3311a4d6c968b135f8e5ee536, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:51.693738 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f690ae-4835-4c40-b520-e14c4a393292 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:51.695454 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3456d7be-6b1b-487b-a1ba-88ee40bb727b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:51.703897 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e2a75-b719-41db-8637-c6374072a53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:51.704770 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e2a75-b719-41db-8637-c6374072a53b admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:51.705651 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e2a75-b719-41db-8637-c6374072a53b admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:51.706895 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3e2a75-b719-41db-8637-c6374072a53b admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:51.707935 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3e2a75-b719-41db-8637-c6374072a53b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:51.710015 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e3e2a75-b719-41db-8637-c6374072a53b admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:51.723513 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 899/1790] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:51 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 20:06:51.728873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e2a75-b719-41db-8637-c6374072a53b admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:51.734978 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 892/1791] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:06:51 2022] PUT /identity/v3/projects/46b77aad0457485693b9a43b85506915/users/48be37e3311a4d6c968b135f8e5ee536/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:51.736528 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64030077-e840-4b88-936c-4768f52adc2e None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:51.737124 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64030077-e840-4b88-936c-4768f52adc2e None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:51.737124 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64030077-e840-4b88-936c-4768f52adc2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:51.749216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:51.759276 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3e2a75-b719-41db-8637-c6374072a53b admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:51.764689 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-64030077-e840-4b88-936c-4768f52adc2e None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:51.766087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-64030077-e840-4b88-936c-4768f52adc2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:51.789004 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:51.860135 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 893/1792] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:51.871102 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a90aa34-2018-4218-8e52-b538e754e0c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:51.871956 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90aa34-2018-4218-8e52-b538e754e0c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:51.872342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90aa34-2018-4218-8e52-b538e754e0c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:51.872719 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90aa34-2018-4218-8e52-b538e754e0c4 admin admin] PATH_INFO: `/v3/projects/70462998e0b94bc183a1603eab19de39/users/5db526ef227b45eba5624c2d111c98b5/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:51.873836 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64030077-e840-4b88-936c-4768f52adc2e None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:51.879138 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-64030077-e840-4b88-936c-4768f52adc2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:51.905850 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a90aa34-2018-4218-8e52-b538e754e0c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70462998e0b94bc183a1603eab19de39, user_id=5db526ef227b45eba5624c2d111c98b5, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:51.927107 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a90aa34-2018-4218-8e52-b538e754e0c4 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:51.956280 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 900/1793] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:06:51 2022] PUT /identity/v3/projects/70462998e0b94bc183a1603eab19de39/users/5db526ef227b45eba5624c2d111c98b5/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:51.972741 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f2f70e-4ef3-4cfb-9ecd-35c3d3ae8ad6 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:51.973047 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f2f70e-4ef3-4cfb-9ecd-35c3d3ae8ad6 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:51.973322 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f2f70e-4ef3-4cfb-9ecd-35c3d3ae8ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:52.014007 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64030077-e840-4b88-936c-4768f52adc2e None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:52.015523 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-25f2f70e-4ef3-4cfb-9ecd-35c3d3ae8ad6 None None] MFA Rules not processed for user `5db526ef227b45eba5624c2d111c98b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:52.017089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-25f2f70e-4ef3-4cfb-9ecd-35c3d3ae8ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:52.115499 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1564e519-249e-46e8-acb7-f3bcdd369463 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:52.116354 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564e519-249e-46e8-acb7-f3bcdd369463 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:52.116683 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564e519-249e-46e8-acb7-f3bcdd369463 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:52.117060 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1564e519-249e-46e8-acb7-f3bcdd369463 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:52.142386 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 901/1794] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:51 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:52.146635 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f2f70e-4ef3-4cfb-9ecd-35c3d3ae8ad6 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:52.251860 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f2f70e-4ef3-4cfb-9ecd-35c3d3ae8ad6 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:52.261639 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7762262b-32ff-47ed-b1d2-d090015ae7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:52.262309 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7762262b-32ff-47ed-b1d2-d090015ae7bf admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:52.262598 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7762262b-32ff-47ed-b1d2-d090015ae7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:52.262873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7762262b-32ff-47ed-b1d2-d090015ae7bf admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:52.265996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7762262b-32ff-47ed-b1d2-d090015ae7bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:52.266399 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1564e519-249e-46e8-acb7-f3bcdd369463 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:52.267948 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1564e519-249e-46e8-acb7-f3bcdd369463 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:52.274568 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7762262b-32ff-47ed-b1d2-d090015ae7bf admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:52.299673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 894/1795] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:06:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:06:52.332242 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 902/1796] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:06:52 2022] POST /identity/v3/projects => generated 393 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:06:52.337161 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1564e519-249e-46e8-acb7-f3bcdd369463 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:52.350669 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1564e519-249e-46e8-acb7-f3bcdd369463 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:52.386873 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=100 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:52.435682 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106720c8-7fa2-413e-898b-9e1898c94637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:52.436321 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106720c8-7fa2-413e-898b-9e1898c94637 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:52.436598 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106720c8-7fa2-413e-898b-9e1898c94637 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:52.436877 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106720c8-7fa2-413e-898b-9e1898c94637 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:52.437528 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106720c8-7fa2-413e-898b-9e1898c94637 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:52.439507 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106720c8-7fa2-413e-898b-9e1898c94637 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:52.463998 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 895/1797] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:06:52 2022] POST /identity/v3/users => generated 303 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:06:52.468248 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7762262b-32ff-47ed-b1d2-d090015ae7bf admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:52.475237 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:52.483612 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7762262b-32ff-47ed-b1d2-d090015ae7bf admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:52.491476 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf8209a-1a19-4d4c-a9de-cdd2642630f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:52.492111 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8209a-1a19-4d4c-a9de-cdd2642630f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:52.492395 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8209a-1a19-4d4c-a9de-cdd2642630f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:52.492670 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8209a-1a19-4d4c-a9de-cdd2642630f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:52.493330 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf8209a-1a19-4d4c-a9de-cdd2642630f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:52.495000 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf8209a-1a19-4d4c-a9de-cdd2642630f4 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:52.505922 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 903/1798] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 20:06:52.510420 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106720c8-7fa2-413e-898b-9e1898c94637 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:52.522111 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106720c8-7fa2-413e-898b-9e1898c94637 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:52.527682 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d8e2d-9165-4095-b9d5-9724152f704a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:52.528356 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d8e2d-9165-4095-b9d5-9724152f704a admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:52.528639 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d8e2d-9165-4095-b9d5-9724152f704a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:52.528943 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d8e2d-9165-4095-b9d5-9724152f704a admin admin] PATH_INFO: `/v3/projects/f2dd7342906b4cc682566fd1e5e9828f/users/458deb10baa84f719d4f7ceadcbe786a/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:52.547388 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d8e2d-9165-4095-b9d5-9724152f704a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2dd7342906b4cc682566fd1e5e9828f, user_id=458deb10baa84f719d4f7ceadcbe786a, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:52.550139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d8e2d-9165-4095-b9d5-9724152f704a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:52.567024 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 896/1799] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:06:52 2022] PUT /identity/v3/projects/f2dd7342906b4cc682566fd1e5e9828f/users/458deb10baa84f719d4f7ceadcbe786a/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:06:52.571107 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf8209a-1a19-4d4c-a9de-cdd2642630f4 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:52.590376 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf8209a-1a19-4d4c-a9de-cdd2642630f4 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:52.692079 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2823fa7c-0433-4532-abc5-ca0cadffe69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:52.692851 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823fa7c-0433-4532-abc5-ca0cadffe69e admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:52.693681 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823fa7c-0433-4532-abc5-ca0cadffe69e admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:52.693957 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2823fa7c-0433-4532-abc5-ca0cadffe69e admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:52.694894 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823fa7c-0433-4532-abc5-ca0cadffe69e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:52.696564 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2823fa7c-0433-4532-abc5-ca0cadffe69e admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:52.713769 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 904/1800] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:06:52.714002 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d8e2d-9165-4095-b9d5-9724152f704a admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:52.732804 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d8e2d-9165-4095-b9d5-9724152f704a admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:52.749718 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30cad8c-092c-4da2-849e-b353fb9656ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:52.750772 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30cad8c-092c-4da2-849e-b353fb9656ad admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:52.751287 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30cad8c-092c-4da2-849e-b353fb9656ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:52.751757 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30cad8c-092c-4da2-849e-b353fb9656ad admin admin] PATH_INFO: `/v3/projects/f2dd7342906b4cc682566fd1e5e9828f/users/458deb10baa84f719d4f7ceadcbe786a/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:52.756620 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30cad8c-092c-4da2-849e-b353fb9656ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2dd7342906b4cc682566fd1e5e9828f, user_id=458deb10baa84f719d4f7ceadcbe786a, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:52.759534 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30cad8c-092c-4da2-849e-b353fb9656ad admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:52.787372 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 897/1801] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:06:52 2022] PUT /identity/v3/projects/f2dd7342906b4cc682566fd1e5e9828f/users/458deb10baa84f719d4f7ceadcbe786a/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:52.805269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e77b57-6508-4fe7-9f0e-e657fe388ec5 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:52.805547 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e77b57-6508-4fe7-9f0e-e657fe388ec5 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:52.805832 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e77b57-6508-4fe7-9f0e-e657fe388ec5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:52.879289 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-65e77b57-6508-4fe7-9f0e-e657fe388ec5 None None] MFA Rules not processed for user `458deb10baa84f719d4f7ceadcbe786a`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:52.880992 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-65e77b57-6508-4fe7-9f0e-e657fe388ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:53.051564 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 905/1802] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:52 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:53.072185 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30cad8c-092c-4da2-849e-b353fb9656ad admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:53.109110 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e77b57-6508-4fe7-9f0e-e657fe388ec5 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:53.110677 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-65e77b57-6508-4fe7-9f0e-e657fe388ec5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:53.210587 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30cad8c-092c-4da2-849e-b353fb9656ad admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:53.222447 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a26debf-d22b-461b-bf2e-22bd2d20c883 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:53.223092 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26debf-d22b-461b-bf2e-22bd2d20c883 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:53.223372 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26debf-d22b-461b-bf2e-22bd2d20c883 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:53.223647 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a26debf-d22b-461b-bf2e-22bd2d20c883 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:53.252602 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a26debf-d22b-461b-bf2e-22bd2d20c883 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:53.254079 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a26debf-d22b-461b-bf2e-22bd2d20c883 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:53.263136 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e77b57-6508-4fe7-9f0e-e657fe388ec5 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:53.281516 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2ee6c0-d862-4621-a583-2d76bf94b3c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:53.282180 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ee6c0-d862-4621-a583-2d76bf94b3c5 service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:53.282485 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ee6c0-d862-4621-a583-2d76bf94b3c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:53.282757 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2ee6c0-d862-4621-a583-2d76bf94b3c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:53.291974 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-2f2ee6c0-d862-4621-a583-2d76bf94b3c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:53.310986 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 898/1803] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:06:53 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:06:53.339050 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:53.398137 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2ee6c0-d862-4621-a583-2d76bf94b3c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:53.400231 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2ee6c0-d862-4621-a583-2d76bf94b3c5 service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:53.426001 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 906/1804] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:06:53 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:06:53.456372 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a26debf-d22b-461b-bf2e-22bd2d20c883 service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:53.464474 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b4d5d-137e-4ee6-a1a9-3f3dbad2c798 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:53.465122 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b4d5d-137e-4ee6-a1a9-3f3dbad2c798 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:53.465535 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427b4d5d-137e-4ee6-a1a9-3f3dbad2c798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:53.480903 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a26debf-d22b-461b-bf2e-22bd2d20c883 service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:53.494331 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd953d6-0248-4b18-a108-1ebc935fb986 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:53.497570 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd953d6-0248-4b18-a108-1ebc935fb986 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:53.497871 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd953d6-0248-4b18-a108-1ebc935fb986 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:53.498272 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd953d6-0248-4b18-a108-1ebc935fb986 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:53.515768 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-427b4d5d-137e-4ee6-a1a9-3f3dbad2c798 None None] MFA Rules not processed for user `48be37e3311a4d6c968b135f8e5ee536`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:53.517769 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-427b4d5d-137e-4ee6-a1a9-3f3dbad2c798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:53.634138 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd953d6-0248-4b18-a108-1ebc935fb986 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:53.634806 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 907/1805] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:53.636234 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd953d6-0248-4b18-a108-1ebc935fb986 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:53.647215 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 899/1806] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:06:53 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:06:53.973622 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b4d5d-137e-4ee6-a1a9-3f3dbad2c798 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:53.986432 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427b4d5d-137e-4ee6-a1a9-3f3dbad2c798 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:53.995419 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965c782-7a04-474f-8d9c-0ba739affff0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:53.996204 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965c782-7a04-474f-8d9c-0ba739affff0 service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:53.996587 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965c782-7a04-474f-8d9c-0ba739affff0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:53.996961 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c965c782-7a04-474f-8d9c-0ba739affff0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:54.015293 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c965c782-7a04-474f-8d9c-0ba739affff0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:54.018733 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c965c782-7a04-474f-8d9c-0ba739affff0 service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:54.033678 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 908/1807] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:06:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 20:06:54.361877 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131507c-3b90-44c0-8a26-db37db12a476 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:54.362323 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131507c-3b90-44c0-8a26-db37db12a476 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:54.362834 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c131507c-3b90-44c0-8a26-db37db12a476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:54.399454 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965c782-7a04-474f-8d9c-0ba739affff0 service cinder] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:54.411170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c965c782-7a04-474f-8d9c-0ba739affff0 service cinder] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:54.415777 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce336baf-f0ff-4645-8d6d-40185e633c92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:54.416879 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce336baf-f0ff-4645-8d6d-40185e633c92 service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:54.417748 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce336baf-f0ff-4645-8d6d-40185e633c92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:54.418186 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce336baf-f0ff-4645-8d6d-40185e633c92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:54.420730 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-ce336baf-f0ff-4645-8d6d-40185e633c92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:54.440395 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-c131507c-3b90-44c0-8a26-db37db12a476 None None] MFA Rules not processed for user `25cd9cc7253a4932ad013f1188995332`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:54.441780 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c131507c-3b90-44c0-8a26-db37db12a476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:54.472094 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:54.528825 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce336baf-f0ff-4645-8d6d-40185e633c92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:54.530666 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce336baf-f0ff-4645-8d6d-40185e633c92 service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:54.542435 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 909/1808] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:06:54 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 20:06:54.582344 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 900/1809] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:54 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:54.592038 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce336baf-f0ff-4645-8d6d-40185e633c92 service cinder] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:54.595122 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-ce336baf-f0ff-4645-8d6d-40185e633c92 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:54.666126 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:54.722881 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce336baf-f0ff-4645-8d6d-40185e633c92 service cinder] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:54.727642 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc892301-a3cb-45fc-9a71-b37c366f9484 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '021216f90d7f4ff6b6eb7a1d86399d99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:54.728489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc892301-a3cb-45fc-9a71-b37c366f9484 service glance] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:54.728608 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc892301-a3cb-45fc-9a71-b37c366f9484 service glance] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:54.728841 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc892301-a3cb-45fc-9a71-b37c366f9484 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:54.730668 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-fc892301-a3cb-45fc-9a71-b37c366f9484 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:54.806979 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc892301-a3cb-45fc-9a71-b37c366f9484 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:54.809178 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc892301-a3cb-45fc-9a71-b37c366f9484 service glance] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:54.822211 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 910/1810] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:06:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:06:55.005272 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61045088-15cf-461a-94be-cf109c835ba4 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:55.006100 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61045088-15cf-461a-94be-cf109c835ba4 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:55.006384 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61045088-15cf-461a-94be-cf109c835ba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:55.037382 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-61045088-15cf-461a-94be-cf109c835ba4 None None] MFA Rules not processed for user `ff5cfe4d31614e2db9059c69116297cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:55.038672 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-61045088-15cf-461a-94be-cf109c835ba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:55.113653 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 901/1811] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Oct 23 20:06:55 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:55.172452 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc892301-a3cb-45fc-9a71-b37c366f9484 service glance] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:55.175049 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-fc892301-a3cb-45fc-9a71-b37c366f9484 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:55.319069 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc892301-a3cb-45fc-9a71-b37c366f9484 service glance] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:55.331326 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313824ae-3e8c-4387-a73f-6b5bdaf6835d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15f603d1e7f54751aa79317e2ea00a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:55.332014 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313824ae-3e8c-4387-a73f-6b5bdaf6835d service swift] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:55.332958 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313824ae-3e8c-4387-a73f-6b5bdaf6835d service swift] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:55.333235 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313824ae-3e8c-4387-a73f-6b5bdaf6835d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:55.347397 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313824ae-3e8c-4387-a73f-6b5bdaf6835d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:55.350315 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313824ae-3e8c-4387-a73f-6b5bdaf6835d service swift] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:55.367522 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 911/1812] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Oct 23 20:06:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 20:06:55.647053 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae42f3-ef4a-4586-89ff-fde58c88e013 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:55.647616 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae42f3-ef4a-4586-89ff-fde58c88e013 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:55.647616 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ae42f3-ef4a-4586-89ff-fde58c88e013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:55.683879 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-11ae42f3-ef4a-4586-89ff-fde58c88e013 None None] MFA Rules not processed for user `48be37e3311a4d6c968b135f8e5ee536`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:55.685675 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-11ae42f3-ef4a-4586-89ff-fde58c88e013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:55.712119 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 902/1813] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:58.724201 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313824ae-3e8c-4387-a73f-6b5bdaf6835d service swift] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:58.843140 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313824ae-3e8c-4387-a73f-6b5bdaf6835d service swift] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:58.866302 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ecbd6-06ff-4812-a375-4ac0730d1480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:58.866302 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ecbd6-06ff-4812-a375-4ac0730d1480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:58.866302 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ecbd6-06ff-4812-a375-4ac0730d1480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:58.866707 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353ecbd6-06ff-4812-a375-4ac0730d1480 admin admin] PATH_INFO: `/v3/users/5db526ef227b45eba5624c2d111c98b5` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:58.867606 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ecbd6-06ff-4812-a375-4ac0730d1480 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5db526ef227b45eba5624c2d111c98b5)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:58.869540 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353ecbd6-06ff-4812-a375-4ac0730d1480 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:58.883866 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:58.980731 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-353ecbd6-06ff-4812-a375-4ac0730d1480 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5db526ef227b45eba5624c2d111c98b5'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:59.013979 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-353ecbd6-06ff-4812-a375-4ac0730d1480 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5db526ef227b45eba5624c2d111c98b5'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:59.029167 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-353ecbd6-06ff-4812-a375-4ac0730d1480 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5db526ef227b45eba5624c2d111c98b5'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:59.049469 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-353ecbd6-06ff-4812-a375-4ac0730d1480 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5db526ef227b45eba5624c2d111c98b5'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:59.050591 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-353ecbd6-06ff-4812-a375-4ac0730d1480 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5db526ef227b45eba5624c2d111c98b5'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:59.065369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 912/1814] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:06:58 2022] DELETE /identity/v3/users/5db526ef227b45eba5624c2d111c98b5 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:06:59.071451 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522547c4-6cbc-42ec-a82f-be22cf72e934 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:59.072030 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522547c4-6cbc-42ec-a82f-be22cf72e934 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:59.072181 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522547c4-6cbc-42ec-a82f-be22cf72e934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:59.112744 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-522547c4-6cbc-42ec-a82f-be22cf72e934 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:59.117475 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-522547c4-6cbc-42ec-a82f-be22cf72e934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:59.260248 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 903/1815] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:59.275657 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ecbd6-06ff-4812-a375-4ac0730d1480 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:59.277179 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-353ecbd6-06ff-4812-a375-4ac0730d1480 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:59.384037 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353ecbd6-06ff-4812-a375-4ac0730d1480 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:59.388631 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a73961-486e-4403-ae83-a70972b93bbf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:59.389300 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a73961-486e-4403-ae83-a70972b93bbf service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:59.389585 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a73961-486e-4403-ae83-a70972b93bbf service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:59.390013 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a73961-486e-4403-ae83-a70972b93bbf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:59.413293 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a73961-486e-4403-ae83-a70972b93bbf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:59.414789 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a73961-486e-4403-ae83-a70972b93bbf service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:59.424913 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522547c4-6cbc-42ec-a82f-be22cf72e934 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:59.427023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-522547c4-6cbc-42ec-a82f-be22cf72e934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:59.453009 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522547c4-6cbc-42ec-a82f-be22cf72e934 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:59.454232 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 913/1816] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:06:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:06:59.459765 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca5f4e-5c0f-43db-9092-95e8e2ff68f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:59.461155 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca5f4e-5c0f-43db-9092-95e8e2ff68f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:59.461649 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca5f4e-5c0f-43db-9092-95e8e2ff68f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:59.462120 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceca5f4e-5c0f-43db-9092-95e8e2ff68f4 admin admin] PATH_INFO: `/v3/users/41a400ff55054e70858c51cfcce3a1c2` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:59.463575 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceca5f4e-5c0f-43db-9092-95e8e2ff68f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41a400ff55054e70858c51cfcce3a1c2)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:59.465712 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceca5f4e-5c0f-43db-9092-95e8e2ff68f4 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:59.526220 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:06:59.536171 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-ceca5f4e-5c0f-43db-9092-95e8e2ff68f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41a400ff55054e70858c51cfcce3a1c2'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:59.554417 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-ceca5f4e-5c0f-43db-9092-95e8e2ff68f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41a400ff55054e70858c51cfcce3a1c2'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:59.564638 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-ceca5f4e-5c0f-43db-9092-95e8e2ff68f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41a400ff55054e70858c51cfcce3a1c2'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:59.565512 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-ceca5f4e-5c0f-43db-9092-95e8e2ff68f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41a400ff55054e70858c51cfcce3a1c2'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:59.576084 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-ceca5f4e-5c0f-43db-9092-95e8e2ff68f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41a400ff55054e70858c51cfcce3a1c2'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:06:59.586804 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 904/1817] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:06:59 2022] DELETE /identity/v3/users/41a400ff55054e70858c51cfcce3a1c2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:06:59.589399 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf0aab-7417-4dc6-8f7a-a7a612b88b27 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:59.589949 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf0aab-7417-4dc6-8f7a-a7a612b88b27 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:59.590231 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf0aab-7417-4dc6-8f7a-a7a612b88b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:59.620354 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-2acf0aab-7417-4dc6-8f7a-a7a612b88b27 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:06:59.621674 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-2acf0aab-7417-4dc6-8f7a-a7a612b88b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:59.735427 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 914/1818] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:06:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:06:59.821269 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca5f4e-5c0f-43db-9092-95e8e2ff68f4 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:06:59.823350 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ceca5f4e-5c0f-43db-9092-95e8e2ff68f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:06:59.931090 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceca5f4e-5c0f-43db-9092-95e8e2ff68f4 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:06:59.935247 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bb3400-a3d4-4605-9173-22b9ef0b4d6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:06:59.935875 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bb3400-a3d4-4605-9173-22b9ef0b4d6d service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:06:59.936157 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bb3400-a3d4-4605-9173-22b9ef0b4d6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:06:59.936430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bb3400-a3d4-4605-9173-22b9ef0b4d6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:06:59.947651 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bb3400-a3d4-4605-9173-22b9ef0b4d6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:06:59.949208 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3bb3400-a3d4-4605-9173-22b9ef0b4d6d service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:06:59.961668 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 905/1819] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:06:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 20:07:00.343323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf0aab-7417-4dc6-8f7a-a7a612b88b27 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:00.344725 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-2acf0aab-7417-4dc6-8f7a-a7a612b88b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:00.363566 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf0aab-7417-4dc6-8f7a-a7a612b88b27 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:00.394568 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af48f4-1800-4d6a-926b-01b759acae25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:00.395516 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af48f4-1800-4d6a-926b-01b759acae25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:00.395971 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af48f4-1800-4d6a-926b-01b759acae25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:00.396408 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af48f4-1800-4d6a-926b-01b759acae25 admin admin] PATH_INFO: `/v3/projects/70462998e0b94bc183a1603eab19de39` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:00.397990 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16af48f4-1800-4d6a-926b-01b759acae25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=70462998e0b94bc183a1603eab19de39)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:00.401368 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16af48f4-1800-4d6a-926b-01b759acae25 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:00.456491 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:00.530112 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-16af48f4-1800-4d6a-926b-01b759acae25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70462998e0b94bc183a1603eab19de39', 'request_id': 'req-16af48f4-1800-4d6a-926b-01b759acae25'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:00.548758 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-16af48f4-1800-4d6a-926b-01b759acae25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70462998e0b94bc183a1603eab19de39', 'request_id': 'req-16af48f4-1800-4d6a-926b-01b759acae25'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:00.568256 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-16af48f4-1800-4d6a-926b-01b759acae25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70462998e0b94bc183a1603eab19de39', 'request_id': 'req-16af48f4-1800-4d6a-926b-01b759acae25'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:00.595631 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-16af48f4-1800-4d6a-926b-01b759acae25 admin admin] The token cache is being invalidate because project 70462998e0b94bc183a1603eab19de39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:07:00.595924 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-16af48f4-1800-4d6a-926b-01b759acae25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:00.596614 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-16af48f4-1800-4d6a-926b-01b759acae25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:00.599843 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 915/1820] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:07:00 2022] DELETE /identity/v3/projects/70462998e0b94bc183a1603eab19de39 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:07:00.602789 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bb3400-a3d4-4605-9173-22b9ef0b4d6d service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:00.617064 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bb3400-a3d4-4605-9173-22b9ef0b4d6d service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:00.750808 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e538ebf1-8564-425c-beba-8daa391e0721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:00.751692 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538ebf1-8564-425c-beba-8daa391e0721 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:00.752910 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538ebf1-8564-425c-beba-8daa391e0721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:00.753195 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538ebf1-8564-425c-beba-8daa391e0721 admin admin] PATH_INFO: `/v3/users/458deb10baa84f719d4f7ceadcbe786a` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:00.754612 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e538ebf1-8564-425c-beba-8daa391e0721 admin admin] RBAC: Authorizing `identity:delete_user(user_id=458deb10baa84f719d4f7ceadcbe786a)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:00.756614 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e538ebf1-8564-425c-beba-8daa391e0721 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:00.916665 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e538ebf1-8564-425c-beba-8daa391e0721 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '458deb10baa84f719d4f7ceadcbe786a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:00.938541 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:00.943542 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e538ebf1-8564-425c-beba-8daa391e0721 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '458deb10baa84f719d4f7ceadcbe786a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:00.966978 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e538ebf1-8564-425c-beba-8daa391e0721 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '458deb10baa84f719d4f7ceadcbe786a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:00.968430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e538ebf1-8564-425c-beba-8daa391e0721 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '458deb10baa84f719d4f7ceadcbe786a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:00.979786 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e538ebf1-8564-425c-beba-8daa391e0721 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '458deb10baa84f719d4f7ceadcbe786a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:01.008334 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 906/1821] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:07:00 2022] DELETE /identity/v3/users/458deb10baa84f719d4f7ceadcbe786a => generated 0 bytes in 403 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:07:01.758654 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af48f4-1800-4d6a-926b-01b759acae25 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:01.760248 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-16af48f4-1800-4d6a-926b-01b759acae25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:01.783493 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af48f4-1800-4d6a-926b-01b759acae25 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:01.891149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dffa4c-d8e8-4a93-afee-3f9e3848378b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:01.891830 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dffa4c-d8e8-4a93-afee-3f9e3848378b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:01.892170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dffa4c-d8e8-4a93-afee-3f9e3848378b admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:01.892514 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dffa4c-d8e8-4a93-afee-3f9e3848378b admin admin] PATH_INFO: `/v3/projects/8d61c457bb9e4d259c25d48cb80761b6` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:01.897103 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dffa4c-d8e8-4a93-afee-3f9e3848378b admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d61c457bb9e4d259c25d48cb80761b6)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:01.899118 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dffa4c-d8e8-4a93-afee-3f9e3848378b admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:02.010000 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c1dffa4c-d8e8-4a93-afee-3f9e3848378b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d61c457bb9e4d259c25d48cb80761b6', 'request_id': 'req-c1dffa4c-d8e8-4a93-afee-3f9e3848378b'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:02.032996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c1dffa4c-d8e8-4a93-afee-3f9e3848378b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d61c457bb9e4d259c25d48cb80761b6', 'request_id': 'req-c1dffa4c-d8e8-4a93-afee-3f9e3848378b'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:02.043138 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c1dffa4c-d8e8-4a93-afee-3f9e3848378b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d61c457bb9e4d259c25d48cb80761b6', 'request_id': 'req-c1dffa4c-d8e8-4a93-afee-3f9e3848378b'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:02.058939 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c1dffa4c-d8e8-4a93-afee-3f9e3848378b admin admin] The token cache is being invalidate because project 8d61c457bb9e4d259c25d48cb80761b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:07:02.059323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c1dffa4c-d8e8-4a93-afee-3f9e3848378b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:02.061752 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c1dffa4c-d8e8-4a93-afee-3f9e3848378b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:02.070008 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 916/1822] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:07:01 2022] DELETE /identity/v3/projects/8d61c457bb9e4d259c25d48cb80761b6 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:07:02.070398 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e538ebf1-8564-425c-beba-8daa391e0721 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:02.072356 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e538ebf1-8564-425c-beba-8daa391e0721 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:02.093042 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e538ebf1-8564-425c-beba-8daa391e0721 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:02.292050 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dffa4c-d8e8-4a93-afee-3f9e3848378b admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:02.293802 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-c1dffa4c-d8e8-4a93-afee-3f9e3848378b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:02.320832 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451a4a39-2ac4-40b2-b0ab-42fdcfc15511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:02.323653 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a4a39-2ac4-40b2-b0ab-42fdcfc15511 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:02.325617 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a4a39-2ac4-40b2-b0ab-42fdcfc15511 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:02.328542 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451a4a39-2ac4-40b2-b0ab-42fdcfc15511 admin admin] PATH_INFO: `/v3/users/071b60bea6ef41bfb58991f8f5d86c51` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:02.331502 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451a4a39-2ac4-40b2-b0ab-42fdcfc15511 admin admin] RBAC: Authorizing `identity:delete_user(user_id=071b60bea6ef41bfb58991f8f5d86c51)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:02.336978 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451a4a39-2ac4-40b2-b0ab-42fdcfc15511 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:02.371050 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dffa4c-d8e8-4a93-afee-3f9e3848378b admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:02.394717 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd562c-aa7e-4ba4-9140-ed6548c51242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:02.395445 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd562c-aa7e-4ba4-9140-ed6548c51242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:02.395863 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd562c-aa7e-4ba4-9140-ed6548c51242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:02.396233 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd562c-aa7e-4ba4-9140-ed6548c51242 admin admin] PATH_INFO: `/v3/projects/f2dd7342906b4cc682566fd1e5e9828f` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:02.397429 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd562c-aa7e-4ba4-9140-ed6548c51242 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2dd7342906b4cc682566fd1e5e9828f)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:02.399414 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd562c-aa7e-4ba4-9140-ed6548c51242 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:02.488344 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:02.509811 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-93cd562c-aa7e-4ba4-9140-ed6548c51242 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2dd7342906b4cc682566fd1e5e9828f', 'request_id': 'req-93cd562c-aa7e-4ba4-9140-ed6548c51242'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:02.521324 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-93cd562c-aa7e-4ba4-9140-ed6548c51242 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2dd7342906b4cc682566fd1e5e9828f', 'request_id': 'req-93cd562c-aa7e-4ba4-9140-ed6548c51242'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:02.550120 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-93cd562c-aa7e-4ba4-9140-ed6548c51242 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2dd7342906b4cc682566fd1e5e9828f', 'request_id': 'req-93cd562c-aa7e-4ba4-9140-ed6548c51242'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:02.571077 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-451a4a39-2ac4-40b2-b0ab-42fdcfc15511 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '071b60bea6ef41bfb58991f8f5d86c51'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:02.575846 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-93cd562c-aa7e-4ba4-9140-ed6548c51242 admin admin] The token cache is being invalidate because project f2dd7342906b4cc682566fd1e5e9828f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:07:02.576135 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-93cd562c-aa7e-4ba4-9140-ed6548c51242 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:02.577821 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-93cd562c-aa7e-4ba4-9140-ed6548c51242 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:02.584931 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 917/1823] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:07:02 2022] DELETE /identity/v3/projects/f2dd7342906b4cc682566fd1e5e9828f => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:07:02.590633 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a34f13-77b0-4bd9-9e58-8684895f1974 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:02.590914 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a34f13-77b0-4bd9-9e58-8684895f1974 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:02.591188 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a34f13-77b0-4bd9-9e58-8684895f1974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:02.636219 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-451a4a39-2ac4-40b2-b0ab-42fdcfc15511 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '071b60bea6ef41bfb58991f8f5d86c51'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:02.637222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-c4a34f13-77b0-4bd9-9e58-8684895f1974 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:07:02.638505 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-c4a34f13-77b0-4bd9-9e58-8684895f1974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:02.678900 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-451a4a39-2ac4-40b2-b0ab-42fdcfc15511 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '071b60bea6ef41bfb58991f8f5d86c51'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:02.684034 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-451a4a39-2ac4-40b2-b0ab-42fdcfc15511 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '071b60bea6ef41bfb58991f8f5d86c51'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:02.740914 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-451a4a39-2ac4-40b2-b0ab-42fdcfc15511 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '071b60bea6ef41bfb58991f8f5d86c51'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:02.752521 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 907/1824] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:07:02 2022] DELETE /identity/v3/users/071b60bea6ef41bfb58991f8f5d86c51 => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:07:02.868558 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 918/1825] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:07:02.876259 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d95e2-04d3-4690-bda2-78a8cd28b4d9 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:02.876639 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d95e2-04d3-4690-bda2-78a8cd28b4d9 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:02.877806 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d95e2-04d3-4690-bda2-78a8cd28b4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:02.948533 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-fb6d95e2-04d3-4690-bda2-78a8cd28b4d9 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:07:02.955228 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-fb6d95e2-04d3-4690-bda2-78a8cd28b4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:03.209812 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:03.244302 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 908/1826] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:07:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:07:03.246779 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a34f13-77b0-4bd9-9e58-8684895f1974 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:03.281755 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a34f13-77b0-4bd9-9e58-8684895f1974 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:03.291510 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f3827-f4fe-42ac-8cf0-7a981bbac418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:03.292227 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f3827-f4fe-42ac-8cf0-7a981bbac418 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:03.292548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f3827-f4fe-42ac-8cf0-7a981bbac418 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:03.292828 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f3827-f4fe-42ac-8cf0-7a981bbac418 admin admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:03.293970 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f3827-f4fe-42ac-8cf0-7a981bbac418 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:07:03.294312 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f3827-f4fe-42ac-8cf0-7a981bbac418 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:03.296716 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04f3827-f4fe-42ac-8cf0-7a981bbac418 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:03.324803 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 919/1827] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:07:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:07:03.326049 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d95e2-04d3-4690-bda2-78a8cd28b4d9 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:03.338315 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d95e2-04d3-4690-bda2-78a8cd28b4d9 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:03.361897 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23786ef7-a72e-4ba7-a4f0-f39b40f7037a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:03.363246 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23786ef7-a72e-4ba7-a4f0-f39b40f7037a admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:03.366289 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23786ef7-a72e-4ba7-a4f0-f39b40f7037a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:03.366602 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23786ef7-a72e-4ba7-a4f0-f39b40f7037a admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:03.367301 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23786ef7-a72e-4ba7-a4f0-f39b40f7037a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:03.369236 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23786ef7-a72e-4ba7-a4f0-f39b40f7037a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:03.463088 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 909/1828] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:07:03 2022] POST /identity/v3/projects => generated 409 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:07:03.467284 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f3827-f4fe-42ac-8cf0-7a981bbac418 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:03.481505 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f3827-f4fe-42ac-8cf0-7a981bbac418 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:03.565522 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d42c9d-42b8-4803-81b2-e15514018e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:03.566327 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d42c9d-42b8-4803-81b2-e15514018e1d admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:03.566685 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d42c9d-42b8-4803-81b2-e15514018e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:03.567032 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d42c9d-42b8-4803-81b2-e15514018e1d admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:03.568294 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d42c9d-42b8-4803-81b2-e15514018e1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:03.570644 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d42c9d-42b8-4803-81b2-e15514018e1d admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:03.607679 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 920/1829] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:07:03 2022] POST /identity/v3/users => generated 305 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:07:03.611879 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23786ef7-a72e-4ba7-a4f0-f39b40f7037a admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:03.631860 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23786ef7-a72e-4ba7-a4f0-f39b40f7037a admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:03.648342 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc65ede-cd10-4d12-9c43-80e4c9c7d70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:03.651768 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc65ede-cd10-4d12-9c43-80e4c9c7d70f admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:03.652380 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc65ede-cd10-4d12-9c43-80e4c9c7d70f admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:03.655161 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc65ede-cd10-4d12-9c43-80e4c9c7d70f admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:03.656371 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc65ede-cd10-4d12-9c43-80e4c9c7d70f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:03.672375 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc65ede-cd10-4d12-9c43-80e4c9c7d70f admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:03.708835 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 910/1830] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:07:03.710552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d42c9d-42b8-4803-81b2-e15514018e1d admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:03.729026 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d42c9d-42b8-4803-81b2-e15514018e1d admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:03.743557 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc65ede-cd10-4d12-9c43-80e4c9c7d70f admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:03.746608 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-1dc65ede-cd10-4d12-9c43-80e4c9c7d70f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:03.747480 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d04858-66f3-4226-9c23-7f353c748c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:03.748124 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d04858-66f3-4226-9c23-7f353c748c2c admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:03.748406 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d04858-66f3-4226-9c23-7f353c748c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:03.748680 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d04858-66f3-4226-9c23-7f353c748c2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:03.749348 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d04858-66f3-4226-9c23-7f353c748c2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:03.750964 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d04858-66f3-4226-9c23-7f353c748c2c admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:03.781969 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc65ede-cd10-4d12-9c43-80e4c9c7d70f admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:03.787847 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4081891-64be-4d33-8922-de9100377ffa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:03.788515 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4081891-64be-4d33-8922-de9100377ffa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:03.788854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4081891-64be-4d33-8922-de9100377ffa admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:03.789226 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4081891-64be-4d33-8922-de9100377ffa admin admin] PATH_INFO: `/v3/projects/7917c227a1554aae95b1c26a3f61309a` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:03.790473 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4081891-64be-4d33-8922-de9100377ffa admin admin] RBAC: Authorizing `identity:delete_project(project_id=7917c227a1554aae95b1c26a3f61309a)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:03.791736 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 921/1831] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:07:03 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:07:03.795564 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4081891-64be-4d33-8922-de9100377ffa admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:03.795923 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d04858-66f3-4226-9c23-7f353c748c2c admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:03.807996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d04858-66f3-4226-9c23-7f353c748c2c admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:03.815693 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92724a96-d9ba-4758-842c-09db438e2a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:03.816510 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92724a96-d9ba-4758-842c-09db438e2a9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:03.816921 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92724a96-d9ba-4758-842c-09db438e2a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:03.817173 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92724a96-d9ba-4758-842c-09db438e2a9e admin admin] PATH_INFO: `/v3/projects/b66442e61be34bf78578a02e6a3d3d60/users/99b5d3f0d0634b6697dd781d268aa7f5/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:03.837037 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92724a96-d9ba-4758-842c-09db438e2a9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b66442e61be34bf78578a02e6a3d3d60, user_id=99b5d3f0d0634b6697dd781d268aa7f5, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:03.839869 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92724a96-d9ba-4758-842c-09db438e2a9e admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:03.860632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 922/1832] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:07:03 2022] PUT /identity/v3/projects/b66442e61be34bf78578a02e6a3d3d60/users/99b5d3f0d0634b6697dd781d268aa7f5/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:07:03.874986 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a95460-09d4-4f76-812d-9aa6a7332d20 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:03.875676 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a95460-09d4-4f76-812d-9aa6a7332d20 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:03.875676 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a95460-09d4-4f76-812d-9aa6a7332d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:03.884491 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d4081891-64be-4d33-8922-de9100377ffa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7917c227a1554aae95b1c26a3f61309a', 'request_id': 'req-d4081891-64be-4d33-8922-de9100377ffa'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:03.898441 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:03.906650 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d4081891-64be-4d33-8922-de9100377ffa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7917c227a1554aae95b1c26a3f61309a', 'request_id': 'req-d4081891-64be-4d33-8922-de9100377ffa'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:03.925292 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d4081891-64be-4d33-8922-de9100377ffa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7917c227a1554aae95b1c26a3f61309a', 'request_id': 'req-d4081891-64be-4d33-8922-de9100377ffa'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:03.932431 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-a4a95460-09d4-4f76-812d-9aa6a7332d20 None None] MFA Rules not processed for user `99b5d3f0d0634b6697dd781d268aa7f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:07:03.943471 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d4081891-64be-4d33-8922-de9100377ffa admin admin] The token cache is being invalidate because project 7917c227a1554aae95b1c26a3f61309a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:07:03.943749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d4081891-64be-4d33-8922-de9100377ffa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:03.944692 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d4081891-64be-4d33-8922-de9100377ffa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:03.946288 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-a4a95460-09d4-4f76-812d-9aa6a7332d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:03.949533 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 911/1833] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:07:03 2022] DELETE /identity/v3/projects/7917c227a1554aae95b1c26a3f61309a => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:07:03.950423 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4081891-64be-4d33-8922-de9100377ffa admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:03.951922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d4081891-64be-4d33-8922-de9100377ffa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:04.059350 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4081891-64be-4d33-8922-de9100377ffa admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:04.079579 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 923/1834] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:07:03 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:07:04.084275 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a95460-09d4-4f76-812d-9aa6a7332d20 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:04.086211 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-a4a95460-09d4-4f76-812d-9aa6a7332d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:04.177891 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:04.195615 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1e93f-26f9-4491-a5b8-4ad3dc370d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:04.196488 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1e93f-26f9-4491-a5b8-4ad3dc370d90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:04.196801 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1e93f-26f9-4491-a5b8-4ad3dc370d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:04.197215 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1e93f-26f9-4491-a5b8-4ad3dc370d90 admin admin] PATH_INFO: `/v3/users/123c7ca57afb4186a8cdb9c4f890b08a` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:04.198631 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1e93f-26f9-4491-a5b8-4ad3dc370d90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=123c7ca57afb4186a8cdb9c4f890b08a)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:04.201655 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1e93f-26f9-4491-a5b8-4ad3dc370d90 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:04.203469 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a95460-09d4-4f76-812d-9aa6a7332d20 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:04.219511 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190929a2-6638-4904-8fab-505554ceb176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:04.220185 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190929a2-6638-4904-8fab-505554ceb176 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:04.220545 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190929a2-6638-4904-8fab-505554ceb176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:04.220855 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190929a2-6638-4904-8fab-505554ceb176 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:04.221589 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190929a2-6638-4904-8fab-505554ceb176 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:04.223775 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190929a2-6638-4904-8fab-505554ceb176 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:04.280021 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 924/1835] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:07:04 2022] POST /identity/v3/projects => generated 407 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:07:04.284505 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190929a2-6638-4904-8fab-505554ceb176 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:04.295926 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-7ca1e93f-26f9-4491-a5b8-4ad3dc370d90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '123c7ca57afb4186a8cdb9c4f890b08a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:04.329754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-7ca1e93f-26f9-4491-a5b8-4ad3dc370d90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '123c7ca57afb4186a8cdb9c4f890b08a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:04.342203 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-7ca1e93f-26f9-4491-a5b8-4ad3dc370d90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '123c7ca57afb4186a8cdb9c4f890b08a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:04.344401 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-7ca1e93f-26f9-4491-a5b8-4ad3dc370d90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '123c7ca57afb4186a8cdb9c4f890b08a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:04.358605 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-7ca1e93f-26f9-4491-a5b8-4ad3dc370d90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '123c7ca57afb4186a8cdb9c4f890b08a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:04.369780 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 912/1836] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:07:03 2022] DELETE /identity/v3/users/123c7ca57afb4186a8cdb9c4f890b08a => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:07:04.385582 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190929a2-6638-4904-8fab-505554ceb176 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:04.416650 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:04.476862 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fde52a-9d9c-4c6d-8481-760637b3644d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:04.477469 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fde52a-9d9c-4c6d-8481-760637b3644d admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:04.477935 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fde52a-9d9c-4c6d-8481-760637b3644d admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:04.478404 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fde52a-9d9c-4c6d-8481-760637b3644d admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:04.479141 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fde52a-9d9c-4c6d-8481-760637b3644d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:04.483116 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fde52a-9d9c-4c6d-8481-760637b3644d admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:04.528969 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1e93f-26f9-4491-a5b8-4ad3dc370d90 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:04.530872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7ca1e93f-26f9-4491-a5b8-4ad3dc370d90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:04.542453 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 925/1837] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:07:04 2022] POST /identity/v3/users => generated 310 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:07:04.556344 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1e93f-26f9-4491-a5b8-4ad3dc370d90 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:04.564204 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5cb93d-2a58-4c5c-ab25-9611360436b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:04.565263 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5cb93d-2a58-4c5c-ab25-9611360436b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:04.565774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5cb93d-2a58-4c5c-ab25-9611360436b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:04.566335 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5cb93d-2a58-4c5c-ab25-9611360436b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:04.567297 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5cb93d-2a58-4c5c-ab25-9611360436b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:04.570364 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5cb93d-2a58-4c5c-ab25-9611360436b7 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:04.588095 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 913/1838] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:07:04.592263 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fde52a-9d9c-4c6d-8481-760637b3644d admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:04.607400 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fde52a-9d9c-4c6d-8481-760637b3644d admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:04.617798 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3fcaf-852e-4e8a-93e3-276f55b80426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:04.618431 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3fcaf-852e-4e8a-93e3-276f55b80426 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:04.618741 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3fcaf-852e-4e8a-93e3-276f55b80426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:04.618983 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3fcaf-852e-4e8a-93e3-276f55b80426 admin admin] PATH_INFO: `/v3/projects/7c721988fd9540359f364872bcf01bd2/users/ef6e2ce410fd498b904fe5b33e3ffdc7/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:04.641273 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc3fcaf-852e-4e8a-93e3-276f55b80426 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c721988fd9540359f364872bcf01bd2, user_id=ef6e2ce410fd498b904fe5b33e3ffdc7, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:04.645211 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc3fcaf-852e-4e8a-93e3-276f55b80426 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:04.667952 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 926/1839] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:07:04 2022] PUT /identity/v3/projects/7c721988fd9540359f364872bcf01bd2/users/ef6e2ce410fd498b904fe5b33e3ffdc7/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:07:04.672199 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5cb93d-2a58-4c5c-ab25-9611360436b7 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:04.688876 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5cb93d-2a58-4c5c-ab25-9611360436b7 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:04.858273 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3300892-699e-446a-9e82-86c801037731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:04.859451 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3300892-699e-446a-9e82-86c801037731 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:04.860040 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3300892-699e-446a-9e82-86c801037731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:04.860501 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3300892-699e-446a-9e82-86c801037731 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:04.861371 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3300892-699e-446a-9e82-86c801037731 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:04.863868 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3300892-699e-446a-9e82-86c801037731 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:04.888814 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3fcaf-852e-4e8a-93e3-276f55b80426 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:04.891008 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-5dc3fcaf-852e-4e8a-93e3-276f55b80426 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:04.913967 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3fcaf-852e-4e8a-93e3-276f55b80426 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:04.915069 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 914/1840] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:07:04.919882 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3300892-699e-446a-9e82-86c801037731 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:04.935773 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ec1bbf-9042-4f5c-a068-e45e4f70724e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:04.939347 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec1bbf-9042-4f5c-a068-e45e4f70724e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:04.939935 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec1bbf-9042-4f5c-a068-e45e4f70724e admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:04.940223 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ec1bbf-9042-4f5c-a068-e45e4f70724e admin admin] PATH_INFO: `/v3/projects/178f0ba912e940ceb156af21b615d9d3` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:04.945752 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ec1bbf-9042-4f5c-a068-e45e4f70724e admin admin] RBAC: Authorizing `identity:delete_project(project_id=178f0ba912e940ceb156af21b615d9d3)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:04.952266 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ec1bbf-9042-4f5c-a068-e45e4f70724e admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:04.969314 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3300892-699e-446a-9e82-86c801037731 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:04.986170 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465eaafb-9ab8-41c8-b9fb-580eb4d913d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:04.989207 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465eaafb-9ab8-41c8-b9fb-580eb4d913d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:04.989486 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465eaafb-9ab8-41c8-b9fb-580eb4d913d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:04.989766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465eaafb-9ab8-41c8-b9fb-580eb4d913d2 admin admin] PATH_INFO: `/v3/projects/7c721988fd9540359f364872bcf01bd2/users/ef6e2ce410fd498b904fe5b33e3ffdc7/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:04.995309 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465eaafb-9ab8-41c8-b9fb-580eb4d913d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c721988fd9540359f364872bcf01bd2, user_id=ef6e2ce410fd498b904fe5b33e3ffdc7, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:05.008501 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465eaafb-9ab8-41c8-b9fb-580eb4d913d2 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:05.088846 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 915/1841] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:07:04 2022] PUT /identity/v3/projects/7c721988fd9540359f364872bcf01bd2/users/ef6e2ce410fd498b904fe5b33e3ffdc7/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:07:05.092150 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268debba-fd45-4ef9-9b4b-a988510c9bcf None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:05.092429 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268debba-fd45-4ef9-9b4b-a988510c9bcf None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:05.092699 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268debba-fd45-4ef9-9b4b-a988510c9bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:05.154949 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-35ec1bbf-9042-4f5c-a068-e45e4f70724e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '178f0ba912e940ceb156af21b615d9d3', 'request_id': 'req-35ec1bbf-9042-4f5c-a068-e45e4f70724e'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:05.157192 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-268debba-fd45-4ef9-9b4b-a988510c9bcf None None] MFA Rules not processed for user `ef6e2ce410fd498b904fe5b33e3ffdc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:07:05.158765 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-268debba-fd45-4ef9-9b4b-a988510c9bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:05.170613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-35ec1bbf-9042-4f5c-a068-e45e4f70724e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '178f0ba912e940ceb156af21b615d9d3', 'request_id': 'req-35ec1bbf-9042-4f5c-a068-e45e4f70724e'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:05.180237 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-35ec1bbf-9042-4f5c-a068-e45e4f70724e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '178f0ba912e940ceb156af21b615d9d3', 'request_id': 'req-35ec1bbf-9042-4f5c-a068-e45e4f70724e'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:05.197064 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-35ec1bbf-9042-4f5c-a068-e45e4f70724e admin admin] The token cache is being invalidate because project 178f0ba912e940ceb156af21b615d9d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:07:05.197255 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-35ec1bbf-9042-4f5c-a068-e45e4f70724e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:05.197921 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-35ec1bbf-9042-4f5c-a068-e45e4f70724e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:05.200655 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 927/1842] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:07:04 2022] DELETE /identity/v3/projects/178f0ba912e940ceb156af21b615d9d3 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:07:05.210814 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a716c3a-8b7e-491e-9736-8efcf397c405 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:05.211297 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a716c3a-8b7e-491e-9736-8efcf397c405 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:05.213892 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a716c3a-8b7e-491e-9736-8efcf397c405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:05.216832 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:05.240970 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-2a716c3a-8b7e-491e-9736-8efcf397c405 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:07:05.242404 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-2a716c3a-8b7e-491e-9736-8efcf397c405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:05.311522 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 916/1843] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:07:05 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:07:05.318797 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268debba-fd45-4ef9-9b4b-a988510c9bcf None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:05.320814 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-268debba-fd45-4ef9-9b4b-a988510c9bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:05.356625 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 928/1844] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:07:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:07:05.369223 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e84e16-cdc6-4e8b-a0a1-0db81e4117ca None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:05.370715 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e84e16-cdc6-4e8b-a0a1-0db81e4117ca None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:05.371094 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e84e16-cdc6-4e8b-a0a1-0db81e4117ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:05.445926 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268debba-fd45-4ef9-9b4b-a988510c9bcf None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:05.450224 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47a4b2-144a-48af-8cf4-a02391f27d42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:05.451506 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47a4b2-144a-48af-8cf4-a02391f27d42 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:05.452132 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47a4b2-144a-48af-8cf4-a02391f27d42 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:05.452743 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e47a4b2-144a-48af-8cf4-a02391f27d42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:05.457096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-86e84e16-cdc6-4e8b-a0a1-0db81e4117ca None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:07:05.458910 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-86e84e16-cdc6-4e8b-a0a1-0db81e4117ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:05.466719 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e47a4b2-144a-48af-8cf4-a02391f27d42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:05.468369 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e47a4b2-144a-48af-8cf4-a02391f27d42 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:05.468893 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 929/1845] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:07:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:07:05.476730 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e84e16-cdc6-4e8b-a0a1-0db81e4117ca None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:05.490891 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e84e16-cdc6-4e8b-a0a1-0db81e4117ca None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:05.496931 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df19d5-86f1-41c1-be46-55f03dcc0729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:05.499314 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 917/1846] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:07:05 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:07:05.513445 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df19d5-86f1-41c1-be46-55f03dcc0729 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:05.514576 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df19d5-86f1-41c1-be46-55f03dcc0729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:05.515904 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7df19d5-86f1-41c1-be46-55f03dcc0729 admin admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:05.517969 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df19d5-86f1-41c1-be46-55f03dcc0729 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:07:05.518789 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df19d5-86f1-41c1-be46-55f03dcc0729 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:05.521844 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7df19d5-86f1-41c1-be46-55f03dcc0729 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:05.523907 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:05.544104 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 930/1847] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:07:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:07:05.545259 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47a4b2-144a-48af-8cf4-a02391f27d42 service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:05.560032 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e47a4b2-144a-48af-8cf4-a02391f27d42 service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:05.575140 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5694c8c-221a-4123-b37b-523f76f510f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:05.576020 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5694c8c-221a-4123-b37b-523f76f510f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:05.576454 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5694c8c-221a-4123-b37b-523f76f510f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:05.576935 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5694c8c-221a-4123-b37b-523f76f510f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:05.578265 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5694c8c-221a-4123-b37b-523f76f510f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:05.580638 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5694c8c-221a-4123-b37b-523f76f510f2 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:05.622882 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 918/1848] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:07:05 2022] POST /identity/v3/projects => generated 367 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:07:05.626745 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df19d5-86f1-41c1-be46-55f03dcc0729 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:05.664230 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7df19d5-86f1-41c1-be46-55f03dcc0729 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:05.742390 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5694c8c-221a-4123-b37b-523f76f510f2 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:05.799723 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c81d54-fc00-41f0-9068-5d2287bc215f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:05.800911 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c81d54-fc00-41f0-9068-5d2287bc215f admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:05.801190 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c81d54-fc00-41f0-9068-5d2287bc215f admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:05.801460 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c81d54-fc00-41f0-9068-5d2287bc215f admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:05.802704 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c81d54-fc00-41f0-9068-5d2287bc215f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:05.807620 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c81d54-fc00-41f0-9068-5d2287bc215f admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:05.840841 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5694c8c-221a-4123-b37b-523f76f510f2 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:05.846835 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d09e58e-7a4d-4d1e-813e-4765a4fe0ccc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:05.847616 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09e58e-7a4d-4d1e-813e-4765a4fe0ccc service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:05.847864 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09e58e-7a4d-4d1e-813e-4765a4fe0ccc service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:05.848198 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d09e58e-7a4d-4d1e-813e-4765a4fe0ccc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:05.851653 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-6d09e58e-7a4d-4d1e-813e-4765a4fe0ccc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:05.862152 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 931/1849] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:07:05 2022] POST /identity/v3/users => generated 284 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:07:05.866848 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c81d54-fc00-41f0-9068-5d2287bc215f admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:05.881749 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c81d54-fc00-41f0-9068-5d2287bc215f admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:05.887011 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a871c98-874f-4115-be4e-d52a01541b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:05.887638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a871c98-874f-4115-be4e-d52a01541b54 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:05.887947 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a871c98-874f-4115-be4e-d52a01541b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:05.888222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a871c98-874f-4115-be4e-d52a01541b54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:05.888877 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a871c98-874f-4115-be4e-d52a01541b54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:05.890821 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a871c98-874f-4115-be4e-d52a01541b54 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:05.907092 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 932/1850] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:07:05 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:07:05.910443 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a871c98-874f-4115-be4e-d52a01541b54 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:05.932929 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a871c98-874f-4115-be4e-d52a01541b54 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:05.942759 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b03516-416d-4ef2-b4b3-32965974ca8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:05.943397 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b03516-416d-4ef2-b4b3-32965974ca8a admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:05.943654 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b03516-416d-4ef2-b4b3-32965974ca8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:05.943933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b03516-416d-4ef2-b4b3-32965974ca8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:05.944837 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b03516-416d-4ef2-b4b3-32965974ca8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:05.958960 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b03516-416d-4ef2-b4b3-32965974ca8a admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:05.984052 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 933/1851] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:07:05 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:07:05.985602 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b03516-416d-4ef2-b4b3-32965974ca8a admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:05.993768 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d09e58e-7a4d-4d1e-813e-4765a4fe0ccc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:05.995027 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d09e58e-7a4d-4d1e-813e-4765a4fe0ccc service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:05.998021 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b03516-416d-4ef2-b4b3-32965974ca8a admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:06.009300 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b366e1b-a4bd-4be9-a629-49bf1b1069d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:06.010590 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b366e1b-a4bd-4be9-a629-49bf1b1069d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:06.011111 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b366e1b-a4bd-4be9-a629-49bf1b1069d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:06.011602 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b366e1b-a4bd-4be9-a629-49bf1b1069d4 admin admin] PATH_INFO: `/v3/projects/884950c9900747699b76fd43700ca8a9/users/517d371acf6543d0ad7c098242a8b217/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:06.012524 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 919/1852] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:07:05 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 271 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 20:07:06.044201 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b366e1b-a4bd-4be9-a629-49bf1b1069d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=884950c9900747699b76fd43700ca8a9, user_id=517d371acf6543d0ad7c098242a8b217, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:06.046942 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b366e1b-a4bd-4be9-a629-49bf1b1069d4 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:06.070597 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 934/1853] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:07:05 2022] PUT /identity/v3/projects/884950c9900747699b76fd43700ca8a9/users/517d371acf6543d0ad7c098242a8b217/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:07:06.076639 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d6c57-fd3c-4cb9-b52f-143385bbe41c None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:06.077452 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d6c57-fd3c-4cb9-b52f-143385bbe41c None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:06.077792 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828d6c57-fd3c-4cb9-b52f-143385bbe41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:06.103924 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:06.156181 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-828d6c57-fd3c-4cb9-b52f-143385bbe41c None None] MFA Rules not processed for user `517d371acf6543d0ad7c098242a8b217`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:07:06.157428 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-828d6c57-fd3c-4cb9-b52f-143385bbe41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:06.362582 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 920/1854] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:07:06 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:07:06.374785 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b366e1b-a4bd-4be9-a629-49bf1b1069d4 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:06.383531 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-6b366e1b-a4bd-4be9-a629-49bf1b1069d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:06.494382 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b366e1b-a4bd-4be9-a629-49bf1b1069d4 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:06.502177 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6433484-77a6-4a71-a479-a5a0c311891f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '021216f90d7f4ff6b6eb7a1d86399d99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:06.504458 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6433484-77a6-4a71-a479-a5a0c311891f service glance] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:06.504821 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6433484-77a6-4a71-a479-a5a0c311891f service glance] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:06.505696 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6433484-77a6-4a71-a479-a5a0c311891f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:06.528933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6433484-77a6-4a71-a479-a5a0c311891f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:06.531003 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6433484-77a6-4a71-a479-a5a0c311891f service glance] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:06.563049 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 935/1855] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:07:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:07:11.475169 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d6c57-fd3c-4cb9-b52f-143385bbe41c None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:11.476693 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-828d6c57-fd3c-4cb9-b52f-143385bbe41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:11.587097 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828d6c57-fd3c-4cb9-b52f-143385bbe41c None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:11.592773 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adca3e8-1b76-407d-8886-4273eb049536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:11.593502 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adca3e8-1b76-407d-8886-4273eb049536 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:11.593853 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adca3e8-1b76-407d-8886-4273eb049536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:11.594268 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adca3e8-1b76-407d-8886-4273eb049536 admin admin] PATH_INFO: `/v3/users/a571b802668942e2b612ef125fe34ea6` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:11.595511 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adca3e8-1b76-407d-8886-4273eb049536 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a571b802668942e2b612ef125fe34ea6)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:11.597949 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adca3e8-1b76-407d-8886-4273eb049536 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:11.660343 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-6adca3e8-1b76-407d-8886-4273eb049536 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a571b802668942e2b612ef125fe34ea6'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:11.676897 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-6adca3e8-1b76-407d-8886-4273eb049536 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a571b802668942e2b612ef125fe34ea6'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:11.693397 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-6adca3e8-1b76-407d-8886-4273eb049536 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a571b802668942e2b612ef125fe34ea6'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:11.694892 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-6adca3e8-1b76-407d-8886-4273eb049536 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a571b802668942e2b612ef125fe34ea6'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:11.710652 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-6adca3e8-1b76-407d-8886-4273eb049536 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a571b802668942e2b612ef125fe34ea6'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:11.724364 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 921/1856] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:07:11 2022] DELETE /identity/v3/users/a571b802668942e2b612ef125fe34ea6 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:07:11.965328 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6433484-77a6-4a71-a479-a5a0c311891f service glance] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:11.982709 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6433484-77a6-4a71-a479-a5a0c311891f service glance] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:12.050870 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:12.106395 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c3fac1-d95e-4f8e-9c61-611174eb9f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:12.107070 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3fac1-d95e-4f8e-9c61-611174eb9f8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:12.107553 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3fac1-d95e-4f8e-9c61-611174eb9f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:12.107844 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c3fac1-d95e-4f8e-9c61-611174eb9f8e admin admin] PATH_INFO: `/v3/projects/dd03c05196b64299bf41b7d4c412070a` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:12.109321 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c3fac1-d95e-4f8e-9c61-611174eb9f8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd03c05196b64299bf41b7d4c412070a)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:12.111306 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c3fac1-d95e-4f8e-9c61-611174eb9f8e admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:12.168391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-e2c3fac1-d95e-4f8e-9c61-611174eb9f8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd03c05196b64299bf41b7d4c412070a', 'request_id': 'req-e2c3fac1-d95e-4f8e-9c61-611174eb9f8e'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:12.177352 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-e2c3fac1-d95e-4f8e-9c61-611174eb9f8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd03c05196b64299bf41b7d4c412070a', 'request_id': 'req-e2c3fac1-d95e-4f8e-9c61-611174eb9f8e'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:12.184352 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-e2c3fac1-d95e-4f8e-9c61-611174eb9f8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd03c05196b64299bf41b7d4c412070a', 'request_id': 'req-e2c3fac1-d95e-4f8e-9c61-611174eb9f8e'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:12.199224 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-e2c3fac1-d95e-4f8e-9c61-611174eb9f8e admin admin] The token cache is being invalidate because project dd03c05196b64299bf41b7d4c412070a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:07:12.199600 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-e2c3fac1-d95e-4f8e-9c61-611174eb9f8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:12.200428 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-e2c3fac1-d95e-4f8e-9c61-611174eb9f8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:12.206328 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 936/1857] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:07:11 2022] DELETE /identity/v3/projects/dd03c05196b64299bf41b7d4c412070a => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:07:12.209981 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f287a-34ac-4a2a-9814-094d6221894e None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:12.210286 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f287a-34ac-4a2a-9814-094d6221894e None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:12.210573 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f287a-34ac-4a2a-9814-094d6221894e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:12.232961 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-595f287a-34ac-4a2a-9814-094d6221894e None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:07:12.234151 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-595f287a-34ac-4a2a-9814-094d6221894e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:12.245695 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:12.309831 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 922/1858] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:07:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:07:12.318042 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549af915-3eea-4513-81c9-53350e250d36 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:12.318365 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549af915-3eea-4513-81c9-53350e250d36 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:12.318631 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549af915-3eea-4513-81c9-53350e250d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:12.347119 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-549af915-3eea-4513-81c9-53350e250d36 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:07:12.348453 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-549af915-3eea-4513-81c9-53350e250d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:12.359340 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 937/1859] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:07:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:07:12.363328 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f287a-34ac-4a2a-9814-094d6221894e None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:12.376198 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f287a-34ac-4a2a-9814-094d6221894e None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:12.384439 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bba3d6-9a9a-4a1d-8a02-7b688c7e379d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:12.385177 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bba3d6-9a9a-4a1d-8a02-7b688c7e379d admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:12.385538 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bba3d6-9a9a-4a1d-8a02-7b688c7e379d admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:12.385896 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bba3d6-9a9a-4a1d-8a02-7b688c7e379d admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:12.386680 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bba3d6-9a9a-4a1d-8a02-7b688c7e379d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:07:12.387047 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bba3d6-9a9a-4a1d-8a02-7b688c7e379d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:12.388732 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bba3d6-9a9a-4a1d-8a02-7b688c7e379d admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:12.406205 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 923/1860] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:07:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 20:07:12.410782 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549af915-3eea-4513-81c9-53350e250d36 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:12.425245 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549af915-3eea-4513-81c9-53350e250d36 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:12.440607 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f99afb-75d5-45f0-91c4-8ad67e802ab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:12.441248 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f99afb-75d5-45f0-91c4-8ad67e802ab4 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:12.441534 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f99afb-75d5-45f0-91c4-8ad67e802ab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:12.441831 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f99afb-75d5-45f0-91c4-8ad67e802ab4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:12.442744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f99afb-75d5-45f0-91c4-8ad67e802ab4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:12.445361 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f99afb-75d5-45f0-91c4-8ad67e802ab4 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:12.478720 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 938/1861] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:07:12 2022] POST /identity/v3/projects => generated 383 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:07:12.479857 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bba3d6-9a9a-4a1d-8a02-7b688c7e379d admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:12.491193 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bba3d6-9a9a-4a1d-8a02-7b688c7e379d admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:12.550577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24e1d8-6328-4f4d-bf3b-5fc7172b48d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:12.551203 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24e1d8-6328-4f4d-bf3b-5fc7172b48d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:12.551520 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24e1d8-6328-4f4d-bf3b-5fc7172b48d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:12.551877 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e24e1d8-6328-4f4d-bf3b-5fc7172b48d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:12.552541 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24e1d8-6328-4f4d-bf3b-5fc7172b48d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:12.554467 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e24e1d8-6328-4f4d-bf3b-5fc7172b48d9 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:12.582890 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 924/1862] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:07:12 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:07:12.586278 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f99afb-75d5-45f0-91c4-8ad67e802ab4 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:12.597608 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f99afb-75d5-45f0-91c4-8ad67e802ab4 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:12.603309 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4046f820-ad86-408b-b161-bdccef1aaf5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:12.604055 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046f820-ad86-408b-b161-bdccef1aaf5f admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:12.604404 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046f820-ad86-408b-b161-bdccef1aaf5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:12.604744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046f820-ad86-408b-b161-bdccef1aaf5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:12.605517 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4046f820-ad86-408b-b161-bdccef1aaf5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:12.607254 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4046f820-ad86-408b-b161-bdccef1aaf5f admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:12.621259 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 939/1863] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:07:12.629288 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24e1d8-6328-4f4d-bf3b-5fc7172b48d9 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:12.642754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e24e1d8-6328-4f4d-bf3b-5fc7172b48d9 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:12.649237 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fd3e69-2ace-4a3f-bbc9-c1364a7a1dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:12.650336 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd3e69-2ace-4a3f-bbc9-c1364a7a1dd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:12.650699 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd3e69-2ace-4a3f-bbc9-c1364a7a1dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:12.654302 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd3e69-2ace-4a3f-bbc9-c1364a7a1dd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:12.654302 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd3e69-2ace-4a3f-bbc9-c1364a7a1dd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:12.658642 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fd3e69-2ace-4a3f-bbc9-c1364a7a1dd7 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:12.671799 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 925/1864] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:07:12.675850 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4046f820-ad86-408b-b161-bdccef1aaf5f admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:12.688549 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4046f820-ad86-408b-b161-bdccef1aaf5f admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:12.693300 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994c543-013c-4b08-9cc3-c32271a52ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:12.693968 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994c543-013c-4b08-9cc3-c32271a52ef7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:12.694273 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994c543-013c-4b08-9cc3-c32271a52ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:12.694821 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994c543-013c-4b08-9cc3-c32271a52ef7 admin admin] PATH_INFO: `/v3/projects/a19f838e496a41c3ac86b6a3951be2c7/users/0153bab0300243068f7def488794886e/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:12.712750 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2994c543-013c-4b08-9cc3-c32271a52ef7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a19f838e496a41c3ac86b6a3951be2c7, user_id=0153bab0300243068f7def488794886e, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:12.715540 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2994c543-013c-4b08-9cc3-c32271a52ef7 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:12.731234 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 940/1865] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:07:12 2022] PUT /identity/v3/projects/a19f838e496a41c3ac86b6a3951be2c7/users/0153bab0300243068f7def488794886e/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:07:12.743057 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25d19a-b8c9-45c0-95c8-70eacc616d57 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:12.743540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25d19a-b8c9-45c0-95c8-70eacc616d57 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:12.743821 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e25d19a-b8c9-45c0-95c8-70eacc616d57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:12.789647 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-2e25d19a-b8c9-45c0-95c8-70eacc616d57 None None] MFA Rules not processed for user `0153bab0300243068f7def488794886e`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:07:12.790926 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2e25d19a-b8c9-45c0-95c8-70eacc616d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:12.838858 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539ba57-2383-4977-85fd-5d9dada760b8 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:12.839153 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539ba57-2383-4977-85fd-5d9dada760b8 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:12.839432 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539ba57-2383-4977-85fd-5d9dada760b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:12.868012 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-4539ba57-2383-4977-85fd-5d9dada760b8 None None] MFA Rules not processed for user `7d0a9c2670794fb4b134e261411fd9cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:07:12.869285 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-4539ba57-2383-4977-85fd-5d9dada760b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:12.874259 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 926/1866] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:07:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:07:12.878930 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e25d19a-b8c9-45c0-95c8-70eacc616d57 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:12.892801 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:12.930533 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 941/1867] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:07:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:07:12.949823 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e25d19a-b8c9-45c0-95c8-70eacc616d57 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:12.954517 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c24b25-ee39-4a76-806f-d1a427a979fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:12.955128 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c24b25-ee39-4a76-806f-d1a427a979fb admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:12.955415 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c24b25-ee39-4a76-806f-d1a427a979fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:12.955690 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c24b25-ee39-4a76-806f-d1a427a979fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:12.956325 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c24b25-ee39-4a76-806f-d1a427a979fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:12.958776 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c24b25-ee39-4a76-806f-d1a427a979fb admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:12.989390 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 927/1868] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:07:12 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:07:12.992266 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539ba57-2383-4977-85fd-5d9dada760b8 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:13.004081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539ba57-2383-4977-85fd-5d9dada760b8 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:13.012902 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:13.071184 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a3b8f8-61cc-4541-ae9f-918d11ee8792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:13.071804 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3b8f8-61cc-4541-ae9f-918d11ee8792 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:13.072086 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3b8f8-61cc-4541-ae9f-918d11ee8792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:13.072364 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3b8f8-61cc-4541-ae9f-918d11ee8792 admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:13.073019 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a3b8f8-61cc-4541-ae9f-918d11ee8792 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:13.074983 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a3b8f8-61cc-4541-ae9f-918d11ee8792 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:13.096446 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 942/1869] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:07:12 2022] POST /identity/v3/users => generated 298 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:07:13.101196 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c24b25-ee39-4a76-806f-d1a427a979fb admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:13.112439 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c24b25-ee39-4a76-806f-d1a427a979fb admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:13.117289 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f22b9-6dfd-4db1-9ef0-d43a7320234b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:13.117908 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f22b9-6dfd-4db1-9ef0-d43a7320234b admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:13.118190 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f22b9-6dfd-4db1-9ef0-d43a7320234b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:13.118504 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187f22b9-6dfd-4db1-9ef0-d43a7320234b admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:13.119292 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187f22b9-6dfd-4db1-9ef0-d43a7320234b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:13.120888 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187f22b9-6dfd-4db1-9ef0-d43a7320234b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:13.131677 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 928/1870] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:07:13.135838 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a3b8f8-61cc-4541-ae9f-918d11ee8792 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:13.146700 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a3b8f8-61cc-4541-ae9f-918d11ee8792 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:13.151678 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd0647e-a312-4083-93cd-097b9bddaab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:13.152341 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0647e-a312-4083-93cd-097b9bddaab7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:13.152778 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0647e-a312-4083-93cd-097b9bddaab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:13.153272 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd0647e-a312-4083-93cd-097b9bddaab7 admin admin] PATH_INFO: `/v3/projects/8bbde8758c3e4e018430937a98b3cf3e/users/f896c11eba074f209d1d541966d3f5b1/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:13.170927 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd0647e-a312-4083-93cd-097b9bddaab7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bbde8758c3e4e018430937a98b3cf3e, user_id=f896c11eba074f209d1d541966d3f5b1, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:13.173590 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd0647e-a312-4083-93cd-097b9bddaab7 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:13.187349 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 943/1871] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:07:13 2022] PUT /identity/v3/projects/8bbde8758c3e4e018430937a98b3cf3e/users/f896c11eba074f209d1d541966d3f5b1/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:07:13.191122 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f22b9-6dfd-4db1-9ef0-d43a7320234b admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:13.205913 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187f22b9-6dfd-4db1-9ef0-d43a7320234b admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:13.266959 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6793e4f9-6e7f-47ba-9d09-3e57b9f586f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:13.268518 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793e4f9-6e7f-47ba-9d09-3e57b9f586f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:13.268803 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793e4f9-6e7f-47ba-9d09-3e57b9f586f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:13.269080 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6793e4f9-6e7f-47ba-9d09-3e57b9f586f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:13.269767 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6793e4f9-6e7f-47ba-9d09-3e57b9f586f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:13.271457 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6793e4f9-6e7f-47ba-9d09-3e57b9f586f6 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:13.282471 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 929/1872] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:07:13.287289 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd0647e-a312-4083-93cd-097b9bddaab7 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:13.298966 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd0647e-a312-4083-93cd-097b9bddaab7 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:13.304384 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fbaeeb-286f-4d10-93e5-bc920147748c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:13.305315 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbaeeb-286f-4d10-93e5-bc920147748c admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:13.305666 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbaeeb-286f-4d10-93e5-bc920147748c admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:13.306010 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fbaeeb-286f-4d10-93e5-bc920147748c admin admin] PATH_INFO: `/v3/projects/8bbde8758c3e4e018430937a98b3cf3e/users/f896c11eba074f209d1d541966d3f5b1/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:13.308061 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fbaeeb-286f-4d10-93e5-bc920147748c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bbde8758c3e4e018430937a98b3cf3e, user_id=f896c11eba074f209d1d541966d3f5b1, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:13.310873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fbaeeb-286f-4d10-93e5-bc920147748c admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:13.322660 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 944/1873] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:07:13 2022] PUT /identity/v3/projects/8bbde8758c3e4e018430937a98b3cf3e/users/f896c11eba074f209d1d541966d3f5b1/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:07:13.332972 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656413c7-556b-471b-ba03-17f65d36bceb None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:13.333934 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656413c7-556b-471b-ba03-17f65d36bceb None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:13.334178 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656413c7-556b-471b-ba03-17f65d36bceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:13.363598 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-656413c7-556b-471b-ba03-17f65d36bceb None None] MFA Rules not processed for user `f896c11eba074f209d1d541966d3f5b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:07:13.365303 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-656413c7-556b-471b-ba03-17f65d36bceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:13.444097 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 930/1874] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:07:13 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:07:13.457602 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fbaeeb-286f-4d10-93e5-bc920147748c admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:13.459384 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-b6fbaeeb-286f-4d10-93e5-bc920147748c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:13.546369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fbaeeb-286f-4d10-93e5-bc920147748c admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:13.550278 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1fd809-52e6-491d-847c-8ee41d46c786 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:13.550891 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fd809-52e6-491d-847c-8ee41d46c786 service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:13.551200 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fd809-52e6-491d-847c-8ee41d46c786 service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:13.551472 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1fd809-52e6-491d-847c-8ee41d46c786 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:13.561461 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1fd809-52e6-491d-847c-8ee41d46c786 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:13.562909 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1fd809-52e6-491d-847c-8ee41d46c786 service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:13.573768 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 945/1875] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:07:13 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:07:13.687324 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656413c7-556b-471b-ba03-17f65d36bceb None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:13.698171 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656413c7-556b-471b-ba03-17f65d36bceb None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:13.710279 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323b7d79-9ced-4d48-ba16-c4984aeab17b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:13.710986 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b7d79-9ced-4d48-ba16-c4984aeab17b service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:13.711267 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b7d79-9ced-4d48-ba16-c4984aeab17b service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:13.711538 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323b7d79-9ced-4d48-ba16-c4984aeab17b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:13.715841 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:13.776255 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323b7d79-9ced-4d48-ba16-c4984aeab17b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:13.777713 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323b7d79-9ced-4d48-ba16-c4984aeab17b service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:13.789004 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 931/1876] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:07:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 20:07:14.092571 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1fd809-52e6-491d-847c-8ee41d46c786 service cinder] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:14.155868 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1fd809-52e6-491d-847c-8ee41d46c786 service cinder] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:14.160519 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f43492-4915-4e88-b1af-e63a67b1a770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:14.161159 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f43492-4915-4e88-b1af-e63a67b1a770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:14.161450 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f43492-4915-4e88-b1af-e63a67b1a770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:14.161724 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f43492-4915-4e88-b1af-e63a67b1a770 admin admin] PATH_INFO: `/v3/users/0153bab0300243068f7def488794886e` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:14.162913 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f43492-4915-4e88-b1af-e63a67b1a770 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0153bab0300243068f7def488794886e)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:14.164835 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f43492-4915-4e88-b1af-e63a67b1a770 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:14.170664 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:14.214015 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-b5f43492-4915-4e88-b1af-e63a67b1a770 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0153bab0300243068f7def488794886e'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:14.227616 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-b5f43492-4915-4e88-b1af-e63a67b1a770 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0153bab0300243068f7def488794886e'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:14.234566 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-b5f43492-4915-4e88-b1af-e63a67b1a770 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0153bab0300243068f7def488794886e'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:14.243000 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-b5f43492-4915-4e88-b1af-e63a67b1a770 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0153bab0300243068f7def488794886e'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:14.243616 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-b5f43492-4915-4e88-b1af-e63a67b1a770 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0153bab0300243068f7def488794886e'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:14.249455 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 946/1877] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:07:14 2022] DELETE /identity/v3/users/0153bab0300243068f7def488794886e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:07:14.254044 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137da73a-74ae-4a32-be25-ba893d60ad32 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:14.254393 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137da73a-74ae-4a32-be25-ba893d60ad32 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:14.254672 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137da73a-74ae-4a32-be25-ba893d60ad32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:14.273987 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-137da73a-74ae-4a32-be25-ba893d60ad32 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:07:14.275289 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-137da73a-74ae-4a32-be25-ba893d60ad32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:14.336423 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 932/1878] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:07:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:07:14.346036 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f43492-4915-4e88-b1af-e63a67b1a770 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:14.347485 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-b5f43492-4915-4e88-b1af-e63a67b1a770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:14.404004 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f43492-4915-4e88-b1af-e63a67b1a770 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:14.407626 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46c04e-b435-45b6-ba23-491d24b1226d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:14.408231 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46c04e-b435-45b6-ba23-491d24b1226d service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:14.408510 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46c04e-b435-45b6-ba23-491d24b1226d service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:14.408781 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f46c04e-b435-45b6-ba23-491d24b1226d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:14.417969 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46c04e-b435-45b6-ba23-491d24b1226d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:14.419434 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f46c04e-b435-45b6-ba23-491d24b1226d service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:14.429459 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 947/1879] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:07:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:07:14.720008 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137da73a-74ae-4a32-be25-ba893d60ad32 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:14.722308 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-137da73a-74ae-4a32-be25-ba893d60ad32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:14.741713 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137da73a-74ae-4a32-be25-ba893d60ad32 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:14.747086 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a865bfc-09f1-4918-8b18-8bece5ced340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:14.747724 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a865bfc-09f1-4918-8b18-8bece5ced340 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:14.748005 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a865bfc-09f1-4918-8b18-8bece5ced340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:14.748326 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a865bfc-09f1-4918-8b18-8bece5ced340 admin admin] PATH_INFO: `/v3/projects/a19f838e496a41c3ac86b6a3951be2c7` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:14.749529 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a865bfc-09f1-4918-8b18-8bece5ced340 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a19f838e496a41c3ac86b6a3951be2c7)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:14.751522 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a865bfc-09f1-4918-8b18-8bece5ced340 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:14.805252 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4a865bfc-09f1-4918-8b18-8bece5ced340 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a19f838e496a41c3ac86b6a3951be2c7', 'request_id': 'req-4a865bfc-09f1-4918-8b18-8bece5ced340'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:14.815935 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4a865bfc-09f1-4918-8b18-8bece5ced340 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a19f838e496a41c3ac86b6a3951be2c7', 'request_id': 'req-4a865bfc-09f1-4918-8b18-8bece5ced340'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:14.822468 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4a865bfc-09f1-4918-8b18-8bece5ced340 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a19f838e496a41c3ac86b6a3951be2c7', 'request_id': 'req-4a865bfc-09f1-4918-8b18-8bece5ced340'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:14.834846 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4a865bfc-09f1-4918-8b18-8bece5ced340 admin admin] The token cache is being invalidate because project a19f838e496a41c3ac86b6a3951be2c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:07:14.835169 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4a865bfc-09f1-4918-8b18-8bece5ced340 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:14.835833 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4a865bfc-09f1-4918-8b18-8bece5ced340 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:14.838217 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 933/1880] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:07:14 2022] DELETE /identity/v3/projects/a19f838e496a41c3ac86b6a3951be2c7 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:07:14.842320 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46c04e-b435-45b6-ba23-491d24b1226d service neutron] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:14.853310 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f46c04e-b435-45b6-ba23-491d24b1226d service neutron] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:14.910946 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1077fb6a-a00d-4365-a5b6-1692ab0b689b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:14.912291 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077fb6a-a00d-4365-a5b6-1692ab0b689b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:14.912419 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077fb6a-a00d-4365-a5b6-1692ab0b689b admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:14.912682 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1077fb6a-a00d-4365-a5b6-1692ab0b689b admin admin] PATH_INFO: `/v3/users/f896c11eba074f209d1d541966d3f5b1` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:14.913948 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1077fb6a-a00d-4365-a5b6-1692ab0b689b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f896c11eba074f209d1d541966d3f5b1)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:14.915997 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1077fb6a-a00d-4365-a5b6-1692ab0b689b admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:14.951639 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:14.967018 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-1077fb6a-a00d-4365-a5b6-1692ab0b689b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f896c11eba074f209d1d541966d3f5b1'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:14.978728 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-1077fb6a-a00d-4365-a5b6-1692ab0b689b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f896c11eba074f209d1d541966d3f5b1'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:14.987405 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-1077fb6a-a00d-4365-a5b6-1692ab0b689b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f896c11eba074f209d1d541966d3f5b1'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:14.996074 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-1077fb6a-a00d-4365-a5b6-1692ab0b689b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f896c11eba074f209d1d541966d3f5b1'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:14.996818 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-1077fb6a-a00d-4365-a5b6-1692ab0b689b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f896c11eba074f209d1d541966d3f5b1'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:15.003045 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 948/1881] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:07:14 2022] DELETE /identity/v3/users/f896c11eba074f209d1d541966d3f5b1 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:07:15.388384 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a865bfc-09f1-4918-8b18-8bece5ced340 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:15.393178 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4a865bfc-09f1-4918-8b18-8bece5ced340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:15.439481 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a865bfc-09f1-4918-8b18-8bece5ced340 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:15.490186 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:15.547386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80d7190-5c43-449d-a74c-4254d1337535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:15.548199 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d7190-5c43-449d-a74c-4254d1337535 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:15.548599 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d7190-5c43-449d-a74c-4254d1337535 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:15.549074 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80d7190-5c43-449d-a74c-4254d1337535 admin admin] PATH_INFO: `/v3/projects/8bbde8758c3e4e018430937a98b3cf3e` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:15.551158 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80d7190-5c43-449d-a74c-4254d1337535 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bbde8758c3e4e018430937a98b3cf3e)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:15.553619 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80d7190-5c43-449d-a74c-4254d1337535 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:15.617837 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-b80d7190-5c43-449d-a74c-4254d1337535 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bbde8758c3e4e018430937a98b3cf3e', 'request_id': 'req-b80d7190-5c43-449d-a74c-4254d1337535'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:15.628001 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-b80d7190-5c43-449d-a74c-4254d1337535 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bbde8758c3e4e018430937a98b3cf3e', 'request_id': 'req-b80d7190-5c43-449d-a74c-4254d1337535'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:15.647702 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-b80d7190-5c43-449d-a74c-4254d1337535 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bbde8758c3e4e018430937a98b3cf3e', 'request_id': 'req-b80d7190-5c43-449d-a74c-4254d1337535'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:15.665395 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-b80d7190-5c43-449d-a74c-4254d1337535 admin admin] The token cache is being invalidate because project 8bbde8758c3e4e018430937a98b3cf3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:07:15.665695 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-b80d7190-5c43-449d-a74c-4254d1337535 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:15.666544 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-b80d7190-5c43-449d-a74c-4254d1337535 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:15.669802 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 934/1882] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:07:15 2022] DELETE /identity/v3/projects/8bbde8758c3e4e018430937a98b3cf3e => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:07:15.675385 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c31d64-e234-4596-979a-20837026c989 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:15.676253 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c31d64-e234-4596-979a-20837026c989 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:15.677367 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c31d64-e234-4596-979a-20837026c989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:15.707586 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-05c31d64-e234-4596-979a-20837026c989 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:07:15.709401 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-05c31d64-e234-4596-979a-20837026c989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:15.804378 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 949/1883] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:07:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:07:15.811827 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cca4f1-ec4f-47d6-9d1e-7a0d2fa0c21a None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:15.812314 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cca4f1-ec4f-47d6-9d1e-7a0d2fa0c21a None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:15.812422 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cca4f1-ec4f-47d6-9d1e-7a0d2fa0c21a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:15.855800 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-b6cca4f1-ec4f-47d6-9d1e-7a0d2fa0c21a None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:07:15.857273 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-b6cca4f1-ec4f-47d6-9d1e-7a0d2fa0c21a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:15.866800 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 935/1884] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:07:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:07:15.871133 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c31d64-e234-4596-979a-20837026c989 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:15.883172 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c31d64-e234-4596-979a-20837026c989 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:15.888526 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76919f5-1965-445f-bd6b-944d529f7e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:15.888991 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76919f5-1965-445f-bd6b-944d529f7e90 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:15.889273 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76919f5-1965-445f-bd6b-944d529f7e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:15.896105 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76919f5-1965-445f-bd6b-944d529f7e90 admin admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:15.896935 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76919f5-1965-445f-bd6b-944d529f7e90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:07:15.897218 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76919f5-1965-445f-bd6b-944d529f7e90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:15.898994 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76919f5-1965-445f-bd6b-944d529f7e90 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:15.921565 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cca4f1-ec4f-47d6-9d1e-7a0d2fa0c21a None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:15.922139 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 950/1885] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:07:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 20:07:15.934532 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cca4f1-ec4f-47d6-9d1e-7a0d2fa0c21a None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:15.941292 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e1ce1-ab5d-4ac4-a4b7-df09b160fef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:15.941921 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e1ce1-ab5d-4ac4-a4b7-df09b160fef2 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:15.942210 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e1ce1-ab5d-4ac4-a4b7-df09b160fef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:15.942567 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090e1ce1-ab5d-4ac4-a4b7-df09b160fef2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:15.943206 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e1ce1-ab5d-4ac4-a4b7-df09b160fef2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:15.945359 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090e1ce1-ab5d-4ac4-a4b7-df09b160fef2 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:15.997435 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 936/1886] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:07:15 2022] POST /identity/v3/projects => generated 373 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:07:16.001275 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76919f5-1965-445f-bd6b-944d529f7e90 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:16.027504 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76919f5-1965-445f-bd6b-944d529f7e90 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:16.090167 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b3c0b-f07b-4325-a8c6-db3f86d214bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:16.090733 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b3c0b-f07b-4325-a8c6-db3f86d214bd admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:16.091031 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b3c0b-f07b-4325-a8c6-db3f86d214bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:16.091290 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b3c0b-f07b-4325-a8c6-db3f86d214bd admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:16.091997 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0b3c0b-f07b-4325-a8c6-db3f86d214bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:16.093990 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0b3c0b-f07b-4325-a8c6-db3f86d214bd admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:16.117420 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 951/1887] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:07:16 2022] POST /identity/v3/users => generated 287 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:07:16.127931 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e1ce1-ab5d-4ac4-a4b7-df09b160fef2 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:16.138881 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090e1ce1-ab5d-4ac4-a4b7-df09b160fef2 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:16.143535 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913dee3-4660-4aab-b2a7-a031ad8c95ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:16.144139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913dee3-4660-4aab-b2a7-a031ad8c95ef admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:16.144423 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913dee3-4660-4aab-b2a7-a031ad8c95ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:16.144697 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913dee3-4660-4aab-b2a7-a031ad8c95ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:16.145332 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913dee3-4660-4aab-b2a7-a031ad8c95ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:16.147576 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913dee3-4660-4aab-b2a7-a031ad8c95ef admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:16.157469 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 937/1888] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:07:16.161392 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b3c0b-f07b-4325-a8c6-db3f86d214bd admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:16.172116 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b3c0b-f07b-4325-a8c6-db3f86d214bd admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:16.177051 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5df0d5-434d-4d7b-8a2d-faed6497689e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:16.177710 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5df0d5-434d-4d7b-8a2d-faed6497689e admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:16.177988 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5df0d5-434d-4d7b-8a2d-faed6497689e admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:16.178294 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5df0d5-434d-4d7b-8a2d-faed6497689e admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:16.178946 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5df0d5-434d-4d7b-8a2d-faed6497689e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:16.180781 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5df0d5-434d-4d7b-8a2d-faed6497689e admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:16.193104 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 952/1889] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:07:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:07:16.197361 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913dee3-4660-4aab-b2a7-a031ad8c95ef admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:16.207650 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913dee3-4660-4aab-b2a7-a031ad8c95ef admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:16.212437 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8760fc-d55e-4c49-96f8-503e44c6e272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:16.213105 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8760fc-d55e-4c49-96f8-503e44c6e272 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:16.213386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8760fc-d55e-4c49-96f8-503e44c6e272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:16.213660 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8760fc-d55e-4c49-96f8-503e44c6e272 admin admin] PATH_INFO: `/v3/projects/ff0a974c04624bf9be19e4ac663d0288/users/00a6c60141d44519b95574eb2b792a02/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:16.229083 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8760fc-d55e-4c49-96f8-503e44c6e272 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff0a974c04624bf9be19e4ac663d0288, user_id=00a6c60141d44519b95574eb2b792a02, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:16.231766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8760fc-d55e-4c49-96f8-503e44c6e272 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:16.245288 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 938/1890] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:07:16 2022] PUT /identity/v3/projects/ff0a974c04624bf9be19e4ac663d0288/users/00a6c60141d44519b95574eb2b792a02/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:07:16.250601 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027dd2f-33c8-4ad5-9765-50e4876fc338 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:16.250884 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027dd2f-33c8-4ad5-9765-50e4876fc338 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:16.251153 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027dd2f-33c8-4ad5-9765-50e4876fc338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:16.269756 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-d027dd2f-33c8-4ad5-9765-50e4876fc338 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:07:16.270976 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-d027dd2f-33c8-4ad5-9765-50e4876fc338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:16.281959 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:16.339118 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 953/1891] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:07:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:07:16.350385 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8760fc-d55e-4c49-96f8-503e44c6e272 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:16.353019 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-8b8760fc-d55e-4c49-96f8-503e44c6e272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:16.436602 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8760fc-d55e-4c49-96f8-503e44c6e272 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:16.440421 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9333f42-1dcf-4e9a-89c1-983a6978bb23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:16.441044 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9333f42-1dcf-4e9a-89c1-983a6978bb23 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:16.441326 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9333f42-1dcf-4e9a-89c1-983a6978bb23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:16.441602 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9333f42-1dcf-4e9a-89c1-983a6978bb23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:16.469254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9333f42-1dcf-4e9a-89c1-983a6978bb23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:16.471037 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9333f42-1dcf-4e9a-89c1-983a6978bb23 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:16.476202 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:16.486131 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 939/1892] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:07:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 20:07:16.776116 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abeeac6-cb09-4e00-92fc-f2edc86e0a3c None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:16.776971 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abeeac6-cb09-4e00-92fc-f2edc86e0a3c None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:16.777387 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abeeac6-cb09-4e00-92fc-f2edc86e0a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:16.801001 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-5abeeac6-cb09-4e00-92fc-f2edc86e0a3c None None] MFA Rules not processed for user `ff5cfe4d31614e2db9059c69116297cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:07:16.802682 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-5abeeac6-cb09-4e00-92fc-f2edc86e0a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:16.924047 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 954/1893] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Oct 23 20:07:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:07:16.945949 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9333f42-1dcf-4e9a-89c1-983a6978bb23 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:16.952604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a9333f42-1dcf-4e9a-89c1-983a6978bb23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:17.120419 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9333f42-1dcf-4e9a-89c1-983a6978bb23 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:17.139556 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab236086-63dc-4932-ad16-bd0337ac4ebe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15f603d1e7f54751aa79317e2ea00a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:17.140199 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab236086-63dc-4932-ad16-bd0337ac4ebe service swift] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:17.140481 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab236086-63dc-4932-ad16-bd0337ac4ebe service swift] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:17.140756 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab236086-63dc-4932-ad16-bd0337ac4ebe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:17.161376 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab236086-63dc-4932-ad16-bd0337ac4ebe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:17.165301 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab236086-63dc-4932-ad16-bd0337ac4ebe service swift] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:17.191204 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 940/1894] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Oct 23 20:07:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 20:07:17.583592 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda0a21-f480-4d08-ab48-ebe62be2780e None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:17.591662 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda0a21-f480-4d08-ab48-ebe62be2780e None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:17.591951 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda0a21-f480-4d08-ab48-ebe62be2780e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:17.662472 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-4eda0a21-f480-4d08-ab48-ebe62be2780e None None] MFA Rules not processed for user `00a6c60141d44519b95574eb2b792a02`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:07:17.663806 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-4eda0a21-f480-4d08-ab48-ebe62be2780e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:17.782120 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 955/1895] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:07:17 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:07:17.788131 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab236086-63dc-4932-ad16-bd0337ac4ebe service swift] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:17.806037 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab236086-63dc-4932-ad16-bd0337ac4ebe service swift] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:17.813888 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df49058-b8d0-4a37-baa0-cd64f5709e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:17.814968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df49058-b8d0-4a37-baa0-cd64f5709e69 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:17.815403 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df49058-b8d0-4a37-baa0-cd64f5709e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:17.816207 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df49058-b8d0-4a37-baa0-cd64f5709e69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:17.816737 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df49058-b8d0-4a37-baa0-cd64f5709e69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:17.825991 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df49058-b8d0-4a37-baa0-cd64f5709e69 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:17.920552 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 941/1896] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:07:17 2022] POST /identity/v3/projects => generated 373 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:07:17.924299 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda0a21-f480-4d08-ab48-ebe62be2780e None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:17.945850 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda0a21-f480-4d08-ab48-ebe62be2780e None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:18.040403 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:18.054147 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a79a99-1369-4d88-a294-07828d5bb084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:18.054987 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a79a99-1369-4d88-a294-07828d5bb084 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:18.055273 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a79a99-1369-4d88-a294-07828d5bb084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:18.055547 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a79a99-1369-4d88-a294-07828d5bb084 admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:18.056842 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a79a99-1369-4d88-a294-07828d5bb084 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:18.062853 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a79a99-1369-4d88-a294-07828d5bb084 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:18.112037 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 956/1897] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:07:17 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:07:18.116629 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df49058-b8d0-4a37-baa0-cd64f5709e69 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:18.133968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df49058-b8d0-4a37-baa0-cd64f5709e69 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:18.143931 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160df7c-e6c9-4e82-aea6-217db124dc5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:18.154954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160df7c-e6c9-4e82-aea6-217db124dc5a admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:18.155700 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160df7c-e6c9-4e82-aea6-217db124dc5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:18.155986 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7160df7c-e6c9-4e82-aea6-217db124dc5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:18.159013 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7160df7c-e6c9-4e82-aea6-217db124dc5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:18.162587 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7160df7c-e6c9-4e82-aea6-217db124dc5a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:18.194293 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 942/1898] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:07:18.198699 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a79a99-1369-4d88-a294-07828d5bb084 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:18.213573 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a79a99-1369-4d88-a294-07828d5bb084 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:18.227079 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ec3e8-11f9-4190-8c31-77b60888a5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:18.227899 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ec3e8-11f9-4190-8c31-77b60888a5f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:18.228292 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ec3e8-11f9-4190-8c31-77b60888a5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:18.228634 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1ec3e8-11f9-4190-8c31-77b60888a5f5 admin admin] PATH_INFO: `/v3/projects/34d72c877caf4740a2ce46c342efb58d/users/01662ffafdd2490d809c5a861bd2d414/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:18.263128 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ec3e8-11f9-4190-8c31-77b60888a5f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34d72c877caf4740a2ce46c342efb58d, user_id=01662ffafdd2490d809c5a861bd2d414, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:18.266869 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1ec3e8-11f9-4190-8c31-77b60888a5f5 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:18.301508 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 957/1899] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:07:18 2022] PUT /identity/v3/projects/34d72c877caf4740a2ce46c342efb58d/users/01662ffafdd2490d809c5a861bd2d414/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:07:18.305654 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160df7c-e6c9-4e82-aea6-217db124dc5a admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:18.325391 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7160df7c-e6c9-4e82-aea6-217db124dc5a admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:18.443928 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0331a722-d931-4bcd-a81d-8d08660f4643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:18.444879 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0331a722-d931-4bcd-a81d-8d08660f4643 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:18.445558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0331a722-d931-4bcd-a81d-8d08660f4643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:18.445981 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0331a722-d931-4bcd-a81d-8d08660f4643 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:18.446901 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0331a722-d931-4bcd-a81d-8d08660f4643 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:18.448899 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0331a722-d931-4bcd-a81d-8d08660f4643 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:18.461975 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 943/1900] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 20:07:18.466125 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ec3e8-11f9-4190-8c31-77b60888a5f5 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:18.478507 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1ec3e8-11f9-4190-8c31-77b60888a5f5 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:18.488025 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3730266-d4b2-440e-9aff-c1a30dbd089d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:18.488719 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3730266-d4b2-440e-9aff-c1a30dbd089d admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:18.489059 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3730266-d4b2-440e-9aff-c1a30dbd089d admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:18.489400 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3730266-d4b2-440e-9aff-c1a30dbd089d admin admin] PATH_INFO: `/v3/projects/34d72c877caf4740a2ce46c342efb58d/users/01662ffafdd2490d809c5a861bd2d414/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:18.491474 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3730266-d4b2-440e-9aff-c1a30dbd089d admin admin] RBAC: Authorizing `identity:create_grant(project_id=34d72c877caf4740a2ce46c342efb58d, user_id=01662ffafdd2490d809c5a861bd2d414, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:18.494502 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3730266-d4b2-440e-9aff-c1a30dbd089d admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:18.507962 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 958/1901] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:07:18 2022] PUT /identity/v3/projects/34d72c877caf4740a2ce46c342efb58d/users/01662ffafdd2490d809c5a861bd2d414/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:07:18.528960 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0331a722-d931-4bcd-a81d-8d08660f4643 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:18.530787 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-0331a722-d931-4bcd-a81d-8d08660f4643 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:18.546237 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0331a722-d931-4bcd-a81d-8d08660f4643 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:18.612833 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:18.633466 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e43df8-7a9e-4fa3-83a5-09b928843b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:18.634426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e43df8-7a9e-4fa3-83a5-09b928843b8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:18.635086 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e43df8-7a9e-4fa3-83a5-09b928843b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:18.635620 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e43df8-7a9e-4fa3-83a5-09b928843b8e admin admin] PATH_INFO: `/v3/users/99b5d3f0d0634b6697dd781d268aa7f5` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:18.637018 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e43df8-7a9e-4fa3-83a5-09b928843b8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=99b5d3f0d0634b6697dd781d268aa7f5)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:18.639288 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e43df8-7a9e-4fa3-83a5-09b928843b8e admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:18.705271 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-36e43df8-7a9e-4fa3-83a5-09b928843b8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99b5d3f0d0634b6697dd781d268aa7f5'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:18.718170 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-36e43df8-7a9e-4fa3-83a5-09b928843b8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99b5d3f0d0634b6697dd781d268aa7f5'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:18.730321 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-36e43df8-7a9e-4fa3-83a5-09b928843b8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99b5d3f0d0634b6697dd781d268aa7f5'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:18.730985 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-36e43df8-7a9e-4fa3-83a5-09b928843b8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99b5d3f0d0634b6697dd781d268aa7f5'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:18.757823 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-36e43df8-7a9e-4fa3-83a5-09b928843b8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99b5d3f0d0634b6697dd781d268aa7f5'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:18.770317 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 944/1902] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:07:18 2022] DELETE /identity/v3/users/99b5d3f0d0634b6697dd781d268aa7f5 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:07:18.775244 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971dcf7-f513-4a76-b5cb-cdd5f6db232d None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:18.775538 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971dcf7-f513-4a76-b5cb-cdd5f6db232d None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:18.775818 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e971dcf7-f513-4a76-b5cb-cdd5f6db232d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:18.796646 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-e971dcf7-f513-4a76-b5cb-cdd5f6db232d None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:07:18.797864 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-e971dcf7-f513-4a76-b5cb-cdd5f6db232d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:18.907583 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 959/1903] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:07:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:07:18.925134 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e43df8-7a9e-4fa3-83a5-09b928843b8e admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:18.927386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-36e43df8-7a9e-4fa3-83a5-09b928843b8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:19.039264 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e43df8-7a9e-4fa3-83a5-09b928843b8e admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:19.044728 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65326375-2fed-477c-a60e-9ef1d60d582d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:19.045823 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65326375-2fed-477c-a60e-9ef1d60d582d service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:19.046159 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65326375-2fed-477c-a60e-9ef1d60d582d service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:19.046516 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65326375-2fed-477c-a60e-9ef1d60d582d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:19.063285 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65326375-2fed-477c-a60e-9ef1d60d582d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:19.065925 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65326375-2fed-477c-a60e-9ef1d60d582d service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:19.085839 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 945/1904] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:07:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:07:19.632786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30d273-bb79-4032-bcd2-ca22999c98a5 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:19.633075 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30d273-bb79-4032-bcd2-ca22999c98a5 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:19.633365 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de30d273-bb79-4032-bcd2-ca22999c98a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:19.689997 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-de30d273-bb79-4032-bcd2-ca22999c98a5 None None] MFA Rules not processed for user `01662ffafdd2490d809c5a861bd2d414`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:07:19.691614 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-de30d273-bb79-4032-bcd2-ca22999c98a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:19.821766 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 960/1905] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:07:19 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:07:19.841545 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65326375-2fed-477c-a60e-9ef1d60d582d service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:19.843040 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-65326375-2fed-477c-a60e-9ef1d60d582d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:19.979612 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65326375-2fed-477c-a60e-9ef1d60d582d service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:19.999245 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f34eff0-5947-4d3c-b2ad-f08981e10e9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:19.999872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34eff0-5947-4d3c-b2ad-f08981e10e9d service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:20.000162 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34eff0-5947-4d3c-b2ad-f08981e10e9d service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:20.000441 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f34eff0-5947-4d3c-b2ad-f08981e10e9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:20.017078 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3f34eff0-5947-4d3c-b2ad-f08981e10e9d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:20.078922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:20.092820 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de30d273-bb79-4032-bcd2-ca22999c98a5 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:20.094498 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-de30d273-bb79-4032-bcd2-ca22999c98a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:20.119419 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de30d273-bb79-4032-bcd2-ca22999c98a5 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:20.133188 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ef124-fc7e-4ddc-8165-dc5587f24706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:20.133993 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ef124-fc7e-4ddc-8165-dc5587f24706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:20.136491 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ef124-fc7e-4ddc-8165-dc5587f24706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:20.136491 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ef124-fc7e-4ddc-8165-dc5587f24706 admin admin] PATH_INFO: `/v3/projects/b66442e61be34bf78578a02e6a3d3d60` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:20.138928 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ef124-fc7e-4ddc-8165-dc5587f24706 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b66442e61be34bf78578a02e6a3d3d60)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:20.143473 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ef124-fc7e-4ddc-8165-dc5587f24706 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:20.164668 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f34eff0-5947-4d3c-b2ad-f08981e10e9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:20.169251 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f34eff0-5947-4d3c-b2ad-f08981e10e9d service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:20.211495 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:20.217299 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 946/1906] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:07:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:07:20.273065 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-b80ef124-fc7e-4ddc-8165-dc5587f24706 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b66442e61be34bf78578a02e6a3d3d60', 'request_id': 'req-b80ef124-fc7e-4ddc-8165-dc5587f24706'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:20.284344 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-b80ef124-fc7e-4ddc-8165-dc5587f24706 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b66442e61be34bf78578a02e6a3d3d60', 'request_id': 'req-b80ef124-fc7e-4ddc-8165-dc5587f24706'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:20.295006 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-b80ef124-fc7e-4ddc-8165-dc5587f24706 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b66442e61be34bf78578a02e6a3d3d60', 'request_id': 'req-b80ef124-fc7e-4ddc-8165-dc5587f24706'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:20.312430 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-b80ef124-fc7e-4ddc-8165-dc5587f24706 admin admin] The token cache is being invalidate because project b66442e61be34bf78578a02e6a3d3d60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:07:20.312727 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-b80ef124-fc7e-4ddc-8165-dc5587f24706 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:20.313399 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-b80ef124-fc7e-4ddc-8165-dc5587f24706 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:20.315308 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 961/1907] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:07:20 2022] DELETE /identity/v3/projects/b66442e61be34bf78578a02e6a3d3d60 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:07:20.319113 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f34eff0-5947-4d3c-b2ad-f08981e10e9d service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:20.333655 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f34eff0-5947-4d3c-b2ad-f08981e10e9d service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:20.429150 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16544ebb-57f1-47e4-9583-e022c1bbb9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:20.429906 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16544ebb-57f1-47e4-9583-e022c1bbb9d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:20.430277 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16544ebb-57f1-47e4-9583-e022c1bbb9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:20.430645 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16544ebb-57f1-47e4-9583-e022c1bbb9d9 admin admin] PATH_INFO: `/v3/users/ef6e2ce410fd498b904fe5b33e3ffdc7` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:20.432530 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16544ebb-57f1-47e4-9583-e022c1bbb9d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef6e2ce410fd498b904fe5b33e3ffdc7)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:20.435470 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16544ebb-57f1-47e4-9583-e022c1bbb9d9 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:20.502815 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-16544ebb-57f1-47e4-9583-e022c1bbb9d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef6e2ce410fd498b904fe5b33e3ffdc7'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:20.525776 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-16544ebb-57f1-47e4-9583-e022c1bbb9d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef6e2ce410fd498b904fe5b33e3ffdc7'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:20.535085 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-16544ebb-57f1-47e4-9583-e022c1bbb9d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef6e2ce410fd498b904fe5b33e3ffdc7'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:20.535779 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-16544ebb-57f1-47e4-9583-e022c1bbb9d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef6e2ce410fd498b904fe5b33e3ffdc7'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:20.543961 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-16544ebb-57f1-47e4-9583-e022c1bbb9d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef6e2ce410fd498b904fe5b33e3ffdc7'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:20.553871 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 947/1908] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:07:20 2022] DELETE /identity/v3/users/ef6e2ce410fd498b904fe5b33e3ffdc7 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:07:20.906993 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ef124-fc7e-4ddc-8165-dc5587f24706 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:20.909241 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-b80ef124-fc7e-4ddc-8165-dc5587f24706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:20.926308 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ef124-fc7e-4ddc-8165-dc5587f24706 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:20.997618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a262356c-9098-4c8f-838d-c3c3174ef694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:20.998326 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262356c-9098-4c8f-838d-c3c3174ef694 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:20.998609 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262356c-9098-4c8f-838d-c3c3174ef694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:20.998884 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a262356c-9098-4c8f-838d-c3c3174ef694 admin admin] PATH_INFO: `/v3/projects/7c721988fd9540359f364872bcf01bd2` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:21.000035 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a262356c-9098-4c8f-838d-c3c3174ef694 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c721988fd9540359f364872bcf01bd2)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:21.001941 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a262356c-9098-4c8f-838d-c3c3174ef694 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:21.059693 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-a262356c-9098-4c8f-838d-c3c3174ef694 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c721988fd9540359f364872bcf01bd2', 'request_id': 'req-a262356c-9098-4c8f-838d-c3c3174ef694'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:21.066993 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-a262356c-9098-4c8f-838d-c3c3174ef694 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c721988fd9540359f364872bcf01bd2', 'request_id': 'req-a262356c-9098-4c8f-838d-c3c3174ef694'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:21.073195 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-a262356c-9098-4c8f-838d-c3c3174ef694 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c721988fd9540359f364872bcf01bd2', 'request_id': 'req-a262356c-9098-4c8f-838d-c3c3174ef694'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:21.085803 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-a262356c-9098-4c8f-838d-c3c3174ef694 admin admin] The token cache is being invalidate because project 7c721988fd9540359f364872bcf01bd2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:07:21.086011 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-a262356c-9098-4c8f-838d-c3c3174ef694 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:21.086726 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-a262356c-9098-4c8f-838d-c3c3174ef694 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:21.088231 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 962/1909] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:07:20 2022] DELETE /identity/v3/projects/7c721988fd9540359f364872bcf01bd2 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:07:21.093235 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69a688-d351-4f72-9595-81a118ee5b7f None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:21.093748 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69a688-d351-4f72-9595-81a118ee5b7f None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:21.093844 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c69a688-d351-4f72-9595-81a118ee5b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:21.114905 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-8c69a688-d351-4f72-9595-81a118ee5b7f None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:07:21.116130 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-8c69a688-d351-4f72-9595-81a118ee5b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:21.162498 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:21.182319 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 948/1910] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:07:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:07:21.192467 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4f4c3-2854-42f9-b507-3c78f05338d4 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:21.192755 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4f4c3-2854-42f9-b507-3c78f05338d4 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:21.193038 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf4f4c3-2854-42f9-b507-3c78f05338d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:21.242721 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-ccf4f4c3-2854-42f9-b507-3c78f05338d4 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:07:21.244069 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-ccf4f4c3-2854-42f9-b507-3c78f05338d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:21.256210 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 963/1911] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:07:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:07:21.257407 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c69a688-d351-4f72-9595-81a118ee5b7f None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:21.271744 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c69a688-d351-4f72-9595-81a118ee5b7f None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:21.277600 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af43991-e0bf-4e6d-811a-750120373fd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:21.278289 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af43991-e0bf-4e6d-811a-750120373fd7 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:21.278528 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af43991-e0bf-4e6d-811a-750120373fd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:21.278911 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af43991-e0bf-4e6d-811a-750120373fd7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:21.279550 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af43991-e0bf-4e6d-811a-750120373fd7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:07:21.279856 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af43991-e0bf-4e6d-811a-750120373fd7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:21.281599 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af43991-e0bf-4e6d-811a-750120373fd7 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:21.301197 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 949/1912] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:07:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 20:07:21.305121 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4f4c3-2854-42f9-b507-3c78f05338d4 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:21.315047 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf4f4c3-2854-42f9-b507-3c78f05338d4 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:21.320762 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a4692-3299-4232-80dc-6dff02558c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:21.321446 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a4692-3299-4232-80dc-6dff02558c78 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:21.321869 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a4692-3299-4232-80dc-6dff02558c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:21.322877 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0a4692-3299-4232-80dc-6dff02558c78 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:21.322980 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a4692-3299-4232-80dc-6dff02558c78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:21.325025 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0a4692-3299-4232-80dc-6dff02558c78 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:21.355659 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 964/1913] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:07:21 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:07:21.360050 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af43991-e0bf-4e6d-811a-750120373fd7 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:21.372296 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af43991-e0bf-4e6d-811a-750120373fd7 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:21.450198 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ba050-0827-49b5-8761-0c9caa81009b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:21.451032 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ba050-0827-49b5-8761-0c9caa81009b admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:21.451318 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ba050-0827-49b5-8761-0c9caa81009b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:21.451593 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ba050-0827-49b5-8761-0c9caa81009b admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:21.452407 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577ba050-0827-49b5-8761-0c9caa81009b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:21.454206 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577ba050-0827-49b5-8761-0c9caa81009b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:21.485413 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 950/1914] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:07:21 2022] POST /identity/v3/users => generated 288 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:07:21.486165 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a4692-3299-4232-80dc-6dff02558c78 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:21.507536 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0a4692-3299-4232-80dc-6dff02558c78 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:21.518627 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eec2a3-c889-41f3-ab84-b925d2bf6684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:21.519280 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eec2a3-c889-41f3-ab84-b925d2bf6684 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:21.519570 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eec2a3-c889-41f3-ab84-b925d2bf6684 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:21.519833 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1eec2a3-c889-41f3-ab84-b925d2bf6684 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:21.520541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eec2a3-c889-41f3-ab84-b925d2bf6684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:21.522232 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1eec2a3-c889-41f3-ab84-b925d2bf6684 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:21.539943 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:21.570667 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ba050-0827-49b5-8761-0c9caa81009b admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:21.570867 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 965/1915] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:07:21.581583 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ba050-0827-49b5-8761-0c9caa81009b admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:21.586925 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe3d386-358b-4f81-888e-b72b11f2d9f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:21.592880 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3d386-358b-4f81-888e-b72b11f2d9f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:21.592880 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3d386-358b-4f81-888e-b72b11f2d9f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:21.592880 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe3d386-358b-4f81-888e-b72b11f2d9f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:21.592880 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe3d386-358b-4f81-888e-b72b11f2d9f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:21.592880 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe3d386-358b-4f81-888e-b72b11f2d9f2 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:21.613684 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eec2a3-c889-41f3-ab84-b925d2bf6684 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:21.621223 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 951/1916] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:07:21 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:07:21.632075 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1eec2a3-c889-41f3-ab84-b925d2bf6684 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:21.637154 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd120067-5041-48ec-86d4-a32109a0b521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:21.637869 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd120067-5041-48ec-86d4-a32109a0b521 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:21.638338 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd120067-5041-48ec-86d4-a32109a0b521 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:21.638624 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd120067-5041-48ec-86d4-a32109a0b521 admin admin] PATH_INFO: `/v3/projects/0743326d087340b4bb51dbb523a86b51/users/addf021f66e3429bb984643f351b319a/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:21.654450 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd120067-5041-48ec-86d4-a32109a0b521 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0743326d087340b4bb51dbb523a86b51, user_id=addf021f66e3429bb984643f351b319a, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:21.657348 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd120067-5041-48ec-86d4-a32109a0b521 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:21.671444 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 966/1917] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:07:21 2022] PUT /identity/v3/projects/0743326d087340b4bb51dbb523a86b51/users/addf021f66e3429bb984643f351b319a/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:07:21.682399 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38400b62-2f60-460e-b0b4-50185a136bcc None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:21.682712 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38400b62-2f60-460e-b0b4-50185a136bcc None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:21.682984 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38400b62-2f60-460e-b0b4-50185a136bcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:21.743516 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-38400b62-2f60-460e-b0b4-50185a136bcc None None] MFA Rules not processed for user `addf021f66e3429bb984643f351b319a`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:07:21.744762 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-38400b62-2f60-460e-b0b4-50185a136bcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:21.813989 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 952/1918] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:07:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:07:21.825621 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd120067-5041-48ec-86d4-a32109a0b521 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:21.827388 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-bd120067-5041-48ec-86d4-a32109a0b521 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:21.888043 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd120067-5041-48ec-86d4-a32109a0b521 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:21.891681 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52e97d6-7e61-4947-8f67-2e6706ba40a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:21.892289 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e97d6-7e61-4947-8f67-2e6706ba40a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:21.892602 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e97d6-7e61-4947-8f67-2e6706ba40a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:21.892876 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52e97d6-7e61-4947-8f67-2e6706ba40a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:21.901796 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52e97d6-7e61-4947-8f67-2e6706ba40a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:21.903334 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52e97d6-7e61-4947-8f67-2e6706ba40a5 service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:21.912663 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 967/1919] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:07:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:07:22.827592 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38400b62-2f60-460e-b0b4-50185a136bcc None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:22.837509 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38400b62-2f60-460e-b0b4-50185a136bcc None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:22.841049 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e40e2b5-d583-4fc3-9b7f-08136c80e9df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:22.841658 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40e2b5-d583-4fc3-9b7f-08136c80e9df service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:22.841931 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40e2b5-d583-4fc3-9b7f-08136c80e9df service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:22.842235 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40e2b5-d583-4fc3-9b7f-08136c80e9df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:22.844023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-8e40e2b5-d583-4fc3-9b7f-08136c80e9df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:22.897975 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:22.921360 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e40e2b5-d583-4fc3-9b7f-08136c80e9df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:22.922830 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e40e2b5-d583-4fc3-9b7f-08136c80e9df service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:22.932199 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 953/1920] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:07:22 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:07:26.281585 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52e97d6-7e61-4947-8f67-2e6706ba40a5 service cinder] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:26.356109 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52e97d6-7e61-4947-8f67-2e6706ba40a5 service cinder] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:26.361000 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb547d9-14bc-4434-9761-9403dce22b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:26.362486 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb547d9-14bc-4434-9761-9403dce22b66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:26.362486 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb547d9-14bc-4434-9761-9403dce22b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:26.362698 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb547d9-14bc-4434-9761-9403dce22b66 admin admin] PATH_INFO: `/v3/users/addf021f66e3429bb984643f351b319a` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:26.367401 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb547d9-14bc-4434-9761-9403dce22b66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=addf021f66e3429bb984643f351b319a)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:26.369391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb547d9-14bc-4434-9761-9403dce22b66 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:26.430365 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-9bb547d9-14bc-4434-9761-9403dce22b66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'addf021f66e3429bb984643f351b319a'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:26.448044 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-9bb547d9-14bc-4434-9761-9403dce22b66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'addf021f66e3429bb984643f351b319a'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:26.455787 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-9bb547d9-14bc-4434-9761-9403dce22b66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'addf021f66e3429bb984643f351b319a'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:26.467969 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-9bb547d9-14bc-4434-9761-9403dce22b66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'addf021f66e3429bb984643f351b319a'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:26.468922 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-9bb547d9-14bc-4434-9761-9403dce22b66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'addf021f66e3429bb984643f351b319a'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:26.471296 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:26.483409 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 968/1921] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:07:26 2022] DELETE /identity/v3/users/addf021f66e3429bb984643f351b319a => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:07:26.488077 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07cadb-9586-492a-950c-aca6e62bb3d6 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:26.490302 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07cadb-9586-492a-950c-aca6e62bb3d6 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:26.491243 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07cadb-9586-492a-950c-aca6e62bb3d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:26.527389 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-ba07cadb-9586-492a-950c-aca6e62bb3d6 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:07:26.528764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ba07cadb-9586-492a-950c-aca6e62bb3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:26.662879 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 954/1922] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:07:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:07:26.674518 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb547d9-14bc-4434-9761-9403dce22b66 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:26.676051 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-9bb547d9-14bc-4434-9761-9403dce22b66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:26.755780 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb547d9-14bc-4434-9761-9403dce22b66 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:26.760087 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a871d31-086e-4cbc-98aa-b77a3d19951e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:26.760717 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a871d31-086e-4cbc-98aa-b77a3d19951e service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:26.760996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a871d31-086e-4cbc-98aa-b77a3d19951e service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:26.761365 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a871d31-086e-4cbc-98aa-b77a3d19951e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:26.773932 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a871d31-086e-4cbc-98aa-b77a3d19951e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:26.775768 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a871d31-086e-4cbc-98aa-b77a3d19951e service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:26.787410 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 969/1923] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:07:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:07:27.115105 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07cadb-9586-492a-950c-aca6e62bb3d6 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:27.116740 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ba07cadb-9586-492a-950c-aca6e62bb3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:27.129150 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07cadb-9586-492a-950c-aca6e62bb3d6 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:27.134538 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f332d8-8a2b-464f-a468-552b25ace826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:27.135200 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f332d8-8a2b-464f-a468-552b25ace826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:27.135481 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f332d8-8a2b-464f-a468-552b25ace826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:27.135759 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f332d8-8a2b-464f-a468-552b25ace826 admin admin] PATH_INFO: `/v3/projects/0743326d087340b4bb51dbb523a86b51` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:27.136923 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f332d8-8a2b-464f-a468-552b25ace826 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0743326d087340b4bb51dbb523a86b51)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:27.138908 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f332d8-8a2b-464f-a468-552b25ace826 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:27.197506 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-54f332d8-8a2b-464f-a468-552b25ace826 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0743326d087340b4bb51dbb523a86b51', 'request_id': 'req-54f332d8-8a2b-464f-a468-552b25ace826'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:27.206788 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-54f332d8-8a2b-464f-a468-552b25ace826 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0743326d087340b4bb51dbb523a86b51', 'request_id': 'req-54f332d8-8a2b-464f-a468-552b25ace826'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:27.213456 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-54f332d8-8a2b-464f-a468-552b25ace826 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0743326d087340b4bb51dbb523a86b51', 'request_id': 'req-54f332d8-8a2b-464f-a468-552b25ace826'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:27.225384 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-54f332d8-8a2b-464f-a468-552b25ace826 admin admin] The token cache is being invalidate because project 0743326d087340b4bb51dbb523a86b51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:07:27.225675 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-54f332d8-8a2b-464f-a468-552b25ace826 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:27.226355 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-54f332d8-8a2b-464f-a468-552b25ace826 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:27.227863 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 955/1924] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:07:27 2022] DELETE /identity/v3/projects/0743326d087340b4bb51dbb523a86b51 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:07:27.232919 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91204e-1e10-4d29-955e-9d2af59a9e62 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:27.233213 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91204e-1e10-4d29-955e-9d2af59a9e62 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:27.233492 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed91204e-1e10-4d29-955e-9d2af59a9e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:27.264612 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-ed91204e-1e10-4d29-955e-9d2af59a9e62 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:07:27.271933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-ed91204e-1e10-4d29-955e-9d2af59a9e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:27.334314 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 970/1925] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:07:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:07:27.342667 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5592e6e-c0e0-45a6-952c-966e1f47e5dc None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:27.343037 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5592e6e-c0e0-45a6-952c-966e1f47e5dc None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:27.343397 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5592e6e-c0e0-45a6-952c-966e1f47e5dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:27.364849 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-c5592e6e-c0e0-45a6-952c-966e1f47e5dc None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:07:27.366185 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c5592e6e-c0e0-45a6-952c-966e1f47e5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:27.374062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 956/1926] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:07:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:07:27.378470 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed91204e-1e10-4d29-955e-9d2af59a9e62 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:27.388786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed91204e-1e10-4d29-955e-9d2af59a9e62 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:27.393404 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94211760-c1bd-4bea-b6b7-366517d077ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:27.394002 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94211760-c1bd-4bea-b6b7-366517d077ab admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:27.394366 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94211760-c1bd-4bea-b6b7-366517d077ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:27.394653 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94211760-c1bd-4bea-b6b7-366517d077ab admin admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:27.395365 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94211760-c1bd-4bea-b6b7-366517d077ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:07:27.396217 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94211760-c1bd-4bea-b6b7-366517d077ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:27.397848 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94211760-c1bd-4bea-b6b7-366517d077ab admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:27.409508 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 971/1927] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:07:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:07:27.414059 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5592e6e-c0e0-45a6-952c-966e1f47e5dc None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:27.424845 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5592e6e-c0e0-45a6-952c-966e1f47e5dc None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:27.429999 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99101d3b-c96d-4fe9-94d3-99c9ec49268f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:27.430650 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99101d3b-c96d-4fe9-94d3-99c9ec49268f admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:27.431002 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99101d3b-c96d-4fe9-94d3-99c9ec49268f admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:27.431287 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99101d3b-c96d-4fe9-94d3-99c9ec49268f admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:27.431952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99101d3b-c96d-4fe9-94d3-99c9ec49268f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:27.433871 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99101d3b-c96d-4fe9-94d3-99c9ec49268f admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:27.469614 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 957/1928] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:07:27 2022] POST /identity/v3/projects => generated 399 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:07:27.470794 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94211760-c1bd-4bea-b6b7-366517d077ab admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:27.480864 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94211760-c1bd-4bea-b6b7-366517d077ab admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:27.545629 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4902be0-dffb-463d-9a11-7d362b33bf2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:27.546404 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4902be0-dffb-463d-9a11-7d362b33bf2e admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:27.546688 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4902be0-dffb-463d-9a11-7d362b33bf2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:27.547101 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4902be0-dffb-463d-9a11-7d362b33bf2e admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:27.547758 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4902be0-dffb-463d-9a11-7d362b33bf2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:27.549683 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4902be0-dffb-463d-9a11-7d362b33bf2e admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:27.592247 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 972/1929] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:07:27 2022] POST /identity/v3/users => generated 300 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:07:27.596297 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99101d3b-c96d-4fe9-94d3-99c9ec49268f admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:27.607650 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99101d3b-c96d-4fe9-94d3-99c9ec49268f admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:27.615797 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e802d625-17a2-4d03-8096-876d7ffe00f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:27.617105 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802d625-17a2-4d03-8096-876d7ffe00f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:27.618598 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802d625-17a2-4d03-8096-876d7ffe00f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:27.619117 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802d625-17a2-4d03-8096-876d7ffe00f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:27.620255 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e802d625-17a2-4d03-8096-876d7ffe00f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:27.622977 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e802d625-17a2-4d03-8096-876d7ffe00f3 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:27.638681 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 958/1930] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:07:27.640844 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4902be0-dffb-463d-9a11-7d362b33bf2e admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:27.652011 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4902be0-dffb-463d-9a11-7d362b33bf2e admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:27.657087 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2e55e-7d55-4bba-a8dc-a0a5443b0759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:27.657671 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2e55e-7d55-4bba-a8dc-a0a5443b0759 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:27.657953 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2e55e-7d55-4bba-a8dc-a0a5443b0759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:27.658230 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2e55e-7d55-4bba-a8dc-a0a5443b0759 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:27.659093 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e2e55e-7d55-4bba-a8dc-a0a5443b0759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:27.660520 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e2e55e-7d55-4bba-a8dc-a0a5443b0759 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:27.678340 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 973/1931] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:07:27 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 20:07:27.684106 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e802d625-17a2-4d03-8096-876d7ffe00f3 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:27.695931 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e802d625-17a2-4d03-8096-876d7ffe00f3 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:27.701676 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ebc685-6adf-437e-b618-1cf13cff9420 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:27.702636 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebc685-6adf-437e-b618-1cf13cff9420 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:27.702920 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebc685-6adf-437e-b618-1cf13cff9420 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:27.703220 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ebc685-6adf-437e-b618-1cf13cff9420 admin admin] PATH_INFO: `/v3/projects/96c2cc20ec0c49d8b408a2b51fb085f5/users/b30dd0d1f2384238817d32b32193c935/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:27.720537 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ebc685-6adf-437e-b618-1cf13cff9420 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96c2cc20ec0c49d8b408a2b51fb085f5, user_id=b30dd0d1f2384238817d32b32193c935, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:27.723335 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ebc685-6adf-437e-b618-1cf13cff9420 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:27.727493 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:27.747778 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 959/1932] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:07:27 2022] PUT /identity/v3/projects/96c2cc20ec0c49d8b408a2b51fb085f5/users/b30dd0d1f2384238817d32b32193c935/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:07:27.822655 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a54fd-bf3b-4123-95b9-6cbb301cacc7 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:27.823120 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a54fd-bf3b-4123-95b9-6cbb301cacc7 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:27.823541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3a54fd-bf3b-4123-95b9-6cbb301cacc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:27.852424 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:27.873323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-7b3a54fd-bf3b-4123-95b9-6cbb301cacc7 None None] MFA Rules not processed for user `b30dd0d1f2384238817d32b32193c935`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:07:27.873323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-7b3a54fd-bf3b-4123-95b9-6cbb301cacc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:27.940278 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 974/1933] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:07:27 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:07:27.957624 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ebc685-6adf-437e-b618-1cf13cff9420 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:27.960929 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f9ebc685-6adf-437e-b618-1cf13cff9420 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:28.078184 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ebc685-6adf-437e-b618-1cf13cff9420 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:28.081825 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7748ee1a-17f3-431b-aafd-a756dc13d185 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:28.082473 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748ee1a-17f3-431b-aafd-a756dc13d185 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:28.083683 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748ee1a-17f3-431b-aafd-a756dc13d185 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:28.083857 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7748ee1a-17f3-431b-aafd-a756dc13d185 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:28.092994 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7748ee1a-17f3-431b-aafd-a756dc13d185 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:28.094784 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7748ee1a-17f3-431b-aafd-a756dc13d185 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:28.103459 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 960/1934] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:07:27 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:07:37.936910 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=29 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:38.099351 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=28 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:41.441410 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a54fd-bf3b-4123-95b9-6cbb301cacc7 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:41.504046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3a54fd-bf3b-4123-95b9-6cbb301cacc7 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:41.508538 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643440c6-2970-4863-8554-a0b0bf5c6af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:41.509160 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643440c6-2970-4863-8554-a0b0bf5c6af5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:41.509437 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643440c6-2970-4863-8554-a0b0bf5c6af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:41.509708 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643440c6-2970-4863-8554-a0b0bf5c6af5 admin admin] PATH_INFO: `/v3/users/b30dd0d1f2384238817d32b32193c935` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:41.510844 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643440c6-2970-4863-8554-a0b0bf5c6af5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b30dd0d1f2384238817d32b32193c935)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:41.512753 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643440c6-2970-4863-8554-a0b0bf5c6af5 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:41.562065 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-643440c6-2970-4863-8554-a0b0bf5c6af5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b30dd0d1f2384238817d32b32193c935'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:41.574498 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-643440c6-2970-4863-8554-a0b0bf5c6af5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b30dd0d1f2384238817d32b32193c935'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:41.581237 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-643440c6-2970-4863-8554-a0b0bf5c6af5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b30dd0d1f2384238817d32b32193c935'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:41.588186 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-643440c6-2970-4863-8554-a0b0bf5c6af5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b30dd0d1f2384238817d32b32193c935'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:41.588862 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-643440c6-2970-4863-8554-a0b0bf5c6af5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b30dd0d1f2384238817d32b32193c935'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:41.594782 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 975/1935] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:07:41 2022] DELETE /identity/v3/users/b30dd0d1f2384238817d32b32193c935 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:07:41.599771 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d7c96a-fa89-4d9e-be62-57db24c7809b None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:41.600234 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d7c96a-fa89-4d9e-be62-57db24c7809b None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:41.600336 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d7c96a-fa89-4d9e-be62-57db24c7809b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:41.619595 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-43d7c96a-fa89-4d9e-be62-57db24c7809b None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:07:41.620805 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-43d7c96a-fa89-4d9e-be62-57db24c7809b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:41.680276 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 961/1936] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:07:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:07:41.690494 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643440c6-2970-4863-8554-a0b0bf5c6af5 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:41.698129 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-643440c6-2970-4863-8554-a0b0bf5c6af5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:41.772317 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643440c6-2970-4863-8554-a0b0bf5c6af5 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:41.776010 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4eb1981-d2a4-45d1-908a-e8192c6b7cf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:41.776624 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb1981-d2a4-45d1-908a-e8192c6b7cf8 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:41.776902 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb1981-d2a4-45d1-908a-e8192c6b7cf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:41.777173 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eb1981-d2a4-45d1-908a-e8192c6b7cf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:41.786450 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eb1981-d2a4-45d1-908a-e8192c6b7cf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:41.787875 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4eb1981-d2a4-45d1-908a-e8192c6b7cf8 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:41.798063 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 976/1937] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:07:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:07:42.113279 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d7c96a-fa89-4d9e-be62-57db24c7809b None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:42.115808 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-43d7c96a-fa89-4d9e-be62-57db24c7809b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:42.127990 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d7c96a-fa89-4d9e-be62-57db24c7809b None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:42.133548 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd5ac96-101d-4694-822d-15fa20f87825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:42.134222 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd5ac96-101d-4694-822d-15fa20f87825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:42.134610 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd5ac96-101d-4694-822d-15fa20f87825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:42.134763 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd5ac96-101d-4694-822d-15fa20f87825 admin admin] PATH_INFO: `/v3/projects/96c2cc20ec0c49d8b408a2b51fb085f5` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:42.135904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd5ac96-101d-4694-822d-15fa20f87825 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96c2cc20ec0c49d8b408a2b51fb085f5)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:42.137849 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd5ac96-101d-4694-822d-15fa20f87825 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:42.184435 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-edd5ac96-101d-4694-822d-15fa20f87825 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96c2cc20ec0c49d8b408a2b51fb085f5', 'request_id': 'req-edd5ac96-101d-4694-822d-15fa20f87825'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:42.191628 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-edd5ac96-101d-4694-822d-15fa20f87825 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96c2cc20ec0c49d8b408a2b51fb085f5', 'request_id': 'req-edd5ac96-101d-4694-822d-15fa20f87825'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:42.198083 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-edd5ac96-101d-4694-822d-15fa20f87825 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96c2cc20ec0c49d8b408a2b51fb085f5', 'request_id': 'req-edd5ac96-101d-4694-822d-15fa20f87825'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:42.209353 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-edd5ac96-101d-4694-822d-15fa20f87825 admin admin] The token cache is being invalidate because project 96c2cc20ec0c49d8b408a2b51fb085f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:07:42.209643 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-edd5ac96-101d-4694-822d-15fa20f87825 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:42.210319 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-edd5ac96-101d-4694-822d-15fa20f87825 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:42.212165 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 962/1938] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:07:42 2022] DELETE /identity/v3/projects/96c2cc20ec0c49d8b408a2b51fb085f5 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:07:45.418762 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205fee78-d922-45ca-b455-ed24362f6aad None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:45.419279 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205fee78-d922-45ca-b455-ed24362f6aad None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:45.419279 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205fee78-d922-45ca-b455-ed24362f6aad None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:45.420604 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 977/1939] 10.222.0.13 () {54 vars in 921 bytes} [Sun Oct 23 20:07:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 20:07:45.428796 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea100a58-983b-4133-8b71-46d1cb456496 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:45.429285 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea100a58-983b-4133-8b71-46d1cb456496 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:45.429378 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea100a58-983b-4133-8b71-46d1cb456496 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:45.449876 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-ea100a58-983b-4133-8b71-46d1cb456496 None None] MFA Rules not processed for user `312489c49a304ca9aacebd9843a4b65f`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:07:45.451340 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ea100a58-983b-4133-8b71-46d1cb456496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:45.530950 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 963/1940] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Oct 23 20:07:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:07:45.550514 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b9d15-bf42-4e0d-a8ac-850db98ede28 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:45.550911 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b9d15-bf42-4e0d-a8ac-850db98ede28 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:45.551295 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8b9d15-bf42-4e0d-a8ac-850db98ede28 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:45.553499 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 978/1941] 10.222.0.13 () {54 vars in 961 bytes} [Sun Oct 23 20:07:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:07:45.559351 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d3990-b666-4d60-8b66-73e43b8a2ddf None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:45.559744 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d3990-b666-4d60-8b66-73e43b8a2ddf None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:45.560131 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4d3990-b666-4d60-8b66-73e43b8a2ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:45.581666 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-0f4d3990-b666-4d60-8b66-73e43b8a2ddf None None] MFA Rules not processed for user `312489c49a304ca9aacebd9843a4b65f`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:07:45.583471 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-0f4d3990-b666-4d60-8b66-73e43b8a2ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:45.593520 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 964/1942] 10.222.0.13 () {58 vars in 1058 bytes} [Sun Oct 23 20:07:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:07:45.600644 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b9d15-bf42-4e0d-a8ac-850db98ede28 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:45.614315 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8b9d15-bf42-4e0d-a8ac-850db98ede28 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:45.620460 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9b088-5ec2-415b-a72c-c0098202876c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312489c49a304ca9aacebd9843a4b65f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:45.621146 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9b088-5ec2-415b-a72c-c0098202876c service placement] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:45.621501 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9b088-5ec2-415b-a72c-c0098202876c service placement] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:45.621851 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f9b088-5ec2-415b-a72c-c0098202876c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:45.637441 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f9b088-5ec2-415b-a72c-c0098202876c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:45.639241 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f9b088-5ec2-415b-a72c-c0098202876c service placement] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:45.650268 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 979/1943] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:07:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 20:07:46.015730 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d3990-b666-4d60-8b66-73e43b8a2ddf None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:46.017510 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-0f4d3990-b666-4d60-8b66-73e43b8a2ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:46.033976 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:46.088255 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4d3990-b666-4d60-8b66-73e43b8a2ddf None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:46.092785 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699bc9e2-7e60-46d8-9a53-5fca1c39c6ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:46.093410 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bc9e2-7e60-46d8-9a53-5fca1c39c6ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:46.093691 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bc9e2-7e60-46d8-9a53-5fca1c39c6ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:46.093966 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bc9e2-7e60-46d8-9a53-5fca1c39c6ed admin admin] PATH_INFO: `/v3/users/00a6c60141d44519b95574eb2b792a02` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:46.095190 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699bc9e2-7e60-46d8-9a53-5fca1c39c6ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=00a6c60141d44519b95574eb2b792a02)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:46.097242 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699bc9e2-7e60-46d8-9a53-5fca1c39c6ed admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:46.144212 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-699bc9e2-7e60-46d8-9a53-5fca1c39c6ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00a6c60141d44519b95574eb2b792a02'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:46.156557 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-699bc9e2-7e60-46d8-9a53-5fca1c39c6ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00a6c60141d44519b95574eb2b792a02'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:46.164088 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-699bc9e2-7e60-46d8-9a53-5fca1c39c6ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00a6c60141d44519b95574eb2b792a02'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:46.164854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-699bc9e2-7e60-46d8-9a53-5fca1c39c6ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00a6c60141d44519b95574eb2b792a02'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:46.171872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-699bc9e2-7e60-46d8-9a53-5fca1c39c6ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00a6c60141d44519b95574eb2b792a02'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:46.177889 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 965/1944] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:07:46 2022] DELETE /identity/v3/users/00a6c60141d44519b95574eb2b792a02 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:07:46.660774 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9b088-5ec2-415b-a72c-c0098202876c service placement] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:46.673178 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f9b088-5ec2-415b-a72c-c0098202876c service placement] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:46.678425 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:46.771025 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587dfd1-71a7-4b6b-8502-cb0f6566741d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:46.782012 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587dfd1-71a7-4b6b-8502-cb0f6566741d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:46.782012 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587dfd1-71a7-4b6b-8502-cb0f6566741d admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:46.782012 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d587dfd1-71a7-4b6b-8502-cb0f6566741d admin admin] PATH_INFO: `/v3/projects/ff0a974c04624bf9be19e4ac663d0288` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:46.782012 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587dfd1-71a7-4b6b-8502-cb0f6566741d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff0a974c04624bf9be19e4ac663d0288)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:46.782012 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d587dfd1-71a7-4b6b-8502-cb0f6566741d admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:46.835265 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-d587dfd1-71a7-4b6b-8502-cb0f6566741d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff0a974c04624bf9be19e4ac663d0288', 'request_id': 'req-d587dfd1-71a7-4b6b-8502-cb0f6566741d'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:46.845528 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-d587dfd1-71a7-4b6b-8502-cb0f6566741d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff0a974c04624bf9be19e4ac663d0288', 'request_id': 'req-d587dfd1-71a7-4b6b-8502-cb0f6566741d'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:46.852373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-d587dfd1-71a7-4b6b-8502-cb0f6566741d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff0a974c04624bf9be19e4ac663d0288', 'request_id': 'req-d587dfd1-71a7-4b6b-8502-cb0f6566741d'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:46.863886 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-d587dfd1-71a7-4b6b-8502-cb0f6566741d admin admin] The token cache is being invalidate because project ff0a974c04624bf9be19e4ac663d0288 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:07:46.864181 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-d587dfd1-71a7-4b6b-8502-cb0f6566741d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:46.864898 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-d587dfd1-71a7-4b6b-8502-cb0f6566741d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:46.866492 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 980/1945] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:07:46 2022] DELETE /identity/v3/projects/ff0a974c04624bf9be19e4ac663d0288 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:07:46.870574 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699bc9e2-7e60-46d8-9a53-5fca1c39c6ed admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:46.871976 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-699bc9e2-7e60-46d8-9a53-5fca1c39c6ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:46.885836 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699bc9e2-7e60-46d8-9a53-5fca1c39c6ed admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:46.946123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698523cf-8009-430f-b110-b6c7714ce8a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:46.946801 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698523cf-8009-430f-b110-b6c7714ce8a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:46.947085 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698523cf-8009-430f-b110-b6c7714ce8a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:46.947362 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698523cf-8009-430f-b110-b6c7714ce8a4 admin admin] PATH_INFO: `/v3/users/01662ffafdd2490d809c5a861bd2d414` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:46.948773 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698523cf-8009-430f-b110-b6c7714ce8a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01662ffafdd2490d809c5a861bd2d414)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:46.951466 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698523cf-8009-430f-b110-b6c7714ce8a4 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:46.995503 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:47.006054 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-698523cf-8009-430f-b110-b6c7714ce8a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01662ffafdd2490d809c5a861bd2d414'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:47.020510 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-698523cf-8009-430f-b110-b6c7714ce8a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01662ffafdd2490d809c5a861bd2d414'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:47.029618 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-698523cf-8009-430f-b110-b6c7714ce8a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01662ffafdd2490d809c5a861bd2d414'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:47.030234 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-698523cf-8009-430f-b110-b6c7714ce8a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01662ffafdd2490d809c5a861bd2d414'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:47.037813 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-698523cf-8009-430f-b110-b6c7714ce8a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01662ffafdd2490d809c5a861bd2d414'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:47.043688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 966/1946] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:07:46 2022] DELETE /identity/v3/users/01662ffafdd2490d809c5a861bd2d414 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:07:47.225902 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587dfd1-71a7-4b6b-8502-cb0f6566741d admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:47.228401 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-d587dfd1-71a7-4b6b-8502-cb0f6566741d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:47.241679 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d587dfd1-71a7-4b6b-8502-cb0f6566741d admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:47.306261 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d819b9-ce8c-4fef-bf58-10f8dda1cf80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:47.307020 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d819b9-ce8c-4fef-bf58-10f8dda1cf80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:47.307309 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d819b9-ce8c-4fef-bf58-10f8dda1cf80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:47.307605 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d819b9-ce8c-4fef-bf58-10f8dda1cf80 admin admin] PATH_INFO: `/v3/projects/34d72c877caf4740a2ce46c342efb58d` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:47.308816 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d819b9-ce8c-4fef-bf58-10f8dda1cf80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34d72c877caf4740a2ce46c342efb58d)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:47.310761 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d819b9-ce8c-4fef-bf58-10f8dda1cf80 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:47.353043 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:47.360492 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-a8d819b9-ce8c-4fef-bf58-10f8dda1cf80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34d72c877caf4740a2ce46c342efb58d', 'request_id': 'req-a8d819b9-ce8c-4fef-bf58-10f8dda1cf80'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:47.368100 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-a8d819b9-ce8c-4fef-bf58-10f8dda1cf80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34d72c877caf4740a2ce46c342efb58d', 'request_id': 'req-a8d819b9-ce8c-4fef-bf58-10f8dda1cf80'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:47.373828 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-a8d819b9-ce8c-4fef-bf58-10f8dda1cf80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34d72c877caf4740a2ce46c342efb58d', 'request_id': 'req-a8d819b9-ce8c-4fef-bf58-10f8dda1cf80'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:47.389431 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-a8d819b9-ce8c-4fef-bf58-10f8dda1cf80 admin admin] The token cache is being invalidate because project 34d72c877caf4740a2ce46c342efb58d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:07:47.389718 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-a8d819b9-ce8c-4fef-bf58-10f8dda1cf80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:47.390489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-a8d819b9-ce8c-4fef-bf58-10f8dda1cf80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:07:47.392087 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 981/1947] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:07:47 2022] DELETE /identity/v3/projects/34d72c877caf4740a2ce46c342efb58d => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:07:47.397617 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f087f-1080-43d9-b5e2-4cd3959b2f85 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:47.397904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f087f-1080-43d9-b5e2-4cd3959b2f85 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:47.398180 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f087f-1080-43d9-b5e2-4cd3959b2f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:47.420501 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-938f087f-1080-43d9-b5e2-4cd3959b2f85 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:07:47.421728 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-938f087f-1080-43d9-b5e2-4cd3959b2f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:47.487723 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 967/1948] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:07:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:07:47.497498 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a94e30-6fa6-4e10-ac0f-9319989203a5 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:47.497786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a94e30-6fa6-4e10-ac0f-9319989203a5 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:47.498062 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a94e30-6fa6-4e10-ac0f-9319989203a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:47.518277 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-f5a94e30-6fa6-4e10-ac0f-9319989203a5 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:07:47.519502 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-f5a94e30-6fa6-4e10-ac0f-9319989203a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:47.528065 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 982/1949] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:07:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:07:47.532346 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f087f-1080-43d9-b5e2-4cd3959b2f85 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:47.542391 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f087f-1080-43d9-b5e2-4cd3959b2f85 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:47.547128 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a07e1b9-8422-40d0-8e02-34d8cf61772f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:47.547735 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07e1b9-8422-40d0-8e02-34d8cf61772f admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:47.548012 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07e1b9-8422-40d0-8e02-34d8cf61772f admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:47.548285 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07e1b9-8422-40d0-8e02-34d8cf61772f admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:47.549031 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a07e1b9-8422-40d0-8e02-34d8cf61772f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:07:47.549336 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a07e1b9-8422-40d0-8e02-34d8cf61772f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:47.550964 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a07e1b9-8422-40d0-8e02-34d8cf61772f admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:47.567048 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 968/1950] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 20:07:47.572224 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a94e30-6fa6-4e10-ac0f-9319989203a5 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:47.581544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a94e30-6fa6-4e10-ac0f-9319989203a5 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:47.586846 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd5108-b536-4c49-bbce-ae9f5bf15d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:47.587447 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd5108-b536-4c49-bbce-ae9f5bf15d7f admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:47.587724 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd5108-b536-4c49-bbce-ae9f5bf15d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:47.588025 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dd5108-b536-4c49-bbce-ae9f5bf15d7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:47.588680 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dd5108-b536-4c49-bbce-ae9f5bf15d7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:47.590735 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dd5108-b536-4c49-bbce-ae9f5bf15d7f admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:47.618079 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 983/1951] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:07:47 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:07:47.622517 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a07e1b9-8422-40d0-8e02-34d8cf61772f admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:47.633282 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a07e1b9-8422-40d0-8e02-34d8cf61772f admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:47.689259 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb361251-d2d0-443c-a988-d2f679899f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:47.689916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb361251-d2d0-443c-a988-d2f679899f8c admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:47.690206 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb361251-d2d0-443c-a988-d2f679899f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:47.690517 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb361251-d2d0-443c-a988-d2f679899f8c admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:47.691173 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb361251-d2d0-443c-a988-d2f679899f8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:47.693204 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb361251-d2d0-443c-a988-d2f679899f8c admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:47.718912 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 969/1952] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:07:47 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:07:47.723233 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd5108-b536-4c49-bbce-ae9f5bf15d7f admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:47.734146 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dd5108-b536-4c49-bbce-ae9f5bf15d7f admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:47.738960 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5552f381-fc39-4fa3-a5aa-932822fe87ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:47.739561 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552f381-fc39-4fa3-a5aa-932822fe87ee admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:47.739838 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552f381-fc39-4fa3-a5aa-932822fe87ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:47.740117 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5552f381-fc39-4fa3-a5aa-932822fe87ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:47.740754 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5552f381-fc39-4fa3-a5aa-932822fe87ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:47.742360 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5552f381-fc39-4fa3-a5aa-932822fe87ee admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:47.752610 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 984/1953] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:07:47.757329 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb361251-d2d0-443c-a988-d2f679899f8c admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:47.767549 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb361251-d2d0-443c-a988-d2f679899f8c admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:47.772758 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6be25-a629-4f09-9ac3-aee2f2c5e0fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:47.773361 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6be25-a629-4f09-9ac3-aee2f2c5e0fd admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:47.773640 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6be25-a629-4f09-9ac3-aee2f2c5e0fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:47.773913 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6be25-a629-4f09-9ac3-aee2f2c5e0fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:47.774587 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c6be25-a629-4f09-9ac3-aee2f2c5e0fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:47.776165 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c6be25-a629-4f09-9ac3-aee2f2c5e0fd admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:47.791141 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 970/1954] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:07:47.792778 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5552f381-fc39-4fa3-a5aa-932822fe87ee admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:47.806178 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5552f381-fc39-4fa3-a5aa-932822fe87ee admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:47.812109 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80663dd-0b89-4856-a2cc-c1ee426ef771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:47.812350 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80663dd-0b89-4856-a2cc-c1ee426ef771 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:47.812740 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80663dd-0b89-4856-a2cc-c1ee426ef771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:47.813156 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80663dd-0b89-4856-a2cc-c1ee426ef771 admin admin] PATH_INFO: `/v3/projects/4cc5317014d7424fb4052b517c7ae1c7/users/cb9d74a88fc64fe887c845394fb42c5d/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:47.833090 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80663dd-0b89-4856-a2cc-c1ee426ef771 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cc5317014d7424fb4052b517c7ae1c7, user_id=cb9d74a88fc64fe887c845394fb42c5d, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:47.836054 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80663dd-0b89-4856-a2cc-c1ee426ef771 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:47.849834 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 985/1955] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:07:47 2022] PUT /identity/v3/projects/4cc5317014d7424fb4052b517c7ae1c7/users/cb9d74a88fc64fe887c845394fb42c5d/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:07:47.860167 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4dc49-7e16-402b-9824-53a610659924 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:47.860449 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4dc49-7e16-402b-9824-53a610659924 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:47.860726 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae4dc49-7e16-402b-9824-53a610659924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:47.891317 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-cae4dc49-7e16-402b-9824-53a610659924 None None] MFA Rules not processed for user `cb9d74a88fc64fe887c845394fb42c5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:07:47.892267 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-cae4dc49-7e16-402b-9824-53a610659924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:47.899567 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:47.971102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 971/1956] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:07:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:07:47.982689 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80663dd-0b89-4856-a2cc-c1ee426ef771 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:07:47.983862 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-f80663dd-0b89-4856-a2cc-c1ee426ef771 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:07:48.066774 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80663dd-0b89-4856-a2cc-c1ee426ef771 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:07:48.072369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5165beb7-738e-4078-910e-88b720458988 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:07:48.072995 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165beb7-738e-4078-910e-88b720458988 service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:07:48.073279 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165beb7-738e-4078-910e-88b720458988 service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:07:48.073558 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5165beb7-738e-4078-910e-88b720458988 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:07:48.086677 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5165beb7-738e-4078-910e-88b720458988 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:07:48.088138 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5165beb7-738e-4078-910e-88b720458988 service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:07:48.097605 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 986/1957] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:07:47 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:07:57.967642 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=24 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:07:58.093315 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=62,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:08:03.042739 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4dc49-7e16-402b-9824-53a610659924 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:08:03.104496 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae4dc49-7e16-402b-9824-53a610659924 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:08:03.108979 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce66abe-6f23-4bed-8251-c362fc9f937c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:08:03.109605 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce66abe-6f23-4bed-8251-c362fc9f937c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:03.109884 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce66abe-6f23-4bed-8251-c362fc9f937c admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:03.110159 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce66abe-6f23-4bed-8251-c362fc9f937c admin admin] PATH_INFO: `/v3/users/cb9d74a88fc64fe887c845394fb42c5d` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:03.111298 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce66abe-6f23-4bed-8251-c362fc9f937c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb9d74a88fc64fe887c845394fb42c5d)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:08:03.113204 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce66abe-6f23-4bed-8251-c362fc9f937c admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:08:03.157743 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-dce66abe-6f23-4bed-8251-c362fc9f937c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb9d74a88fc64fe887c845394fb42c5d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:08:03.170186 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-dce66abe-6f23-4bed-8251-c362fc9f937c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb9d74a88fc64fe887c845394fb42c5d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:08:03.181528 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-dce66abe-6f23-4bed-8251-c362fc9f937c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb9d74a88fc64fe887c845394fb42c5d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:08:03.182157 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-dce66abe-6f23-4bed-8251-c362fc9f937c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb9d74a88fc64fe887c845394fb42c5d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:08:03.189885 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-dce66abe-6f23-4bed-8251-c362fc9f937c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb9d74a88fc64fe887c845394fb42c5d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:08:03.196666 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 972/1958] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:08:03 2022] DELETE /identity/v3/users/cb9d74a88fc64fe887c845394fb42c5d => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:08:03.202180 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3c204-a695-4681-970c-765558f0dbfc None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:03.202675 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3c204-a695-4681-970c-765558f0dbfc None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:03.202772 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff3c204-a695-4681-970c-765558f0dbfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:03.222455 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-eff3c204-a695-4681-970c-765558f0dbfc None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:08:03.223672 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-eff3c204-a695-4681-970c-765558f0dbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:08:03.284676 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 987/1959] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:08:03.294643 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce66abe-6f23-4bed-8251-c362fc9f937c admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:08:03.296045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-dce66abe-6f23-4bed-8251-c362fc9f937c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:08:03.353357 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce66abe-6f23-4bed-8251-c362fc9f937c admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:08:03.356929 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479664e4-7d30-4ddd-a4f9-16be0c83d205 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:08:03.357527 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479664e4-7d30-4ddd-a4f9-16be0c83d205 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:03.357827 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479664e4-7d30-4ddd-a4f9-16be0c83d205 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:03.358080 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479664e4-7d30-4ddd-a4f9-16be0c83d205 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:03.367324 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479664e4-7d30-4ddd-a4f9-16be0c83d205 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:08:03.368736 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479664e4-7d30-4ddd-a4f9-16be0c83d205 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:08:03.378007 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 973/1960] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:08:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:08:03.644106 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff3c204-a695-4681-970c-765558f0dbfc None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:08:03.655011 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-eff3c204-a695-4681-970c-765558f0dbfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:08:03.658433 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff3c204-a695-4681-970c-765558f0dbfc None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:08:03.663356 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4470ba2-eb66-4d9e-8081-9ebf708e2b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:08:03.663999 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4470ba2-eb66-4d9e-8081-9ebf708e2b17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:03.664282 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4470ba2-eb66-4d9e-8081-9ebf708e2b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:03.664560 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4470ba2-eb66-4d9e-8081-9ebf708e2b17 admin admin] PATH_INFO: `/v3/projects/4cc5317014d7424fb4052b517c7ae1c7` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:03.665695 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4470ba2-eb66-4d9e-8081-9ebf708e2b17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cc5317014d7424fb4052b517c7ae1c7)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:08:03.667680 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4470ba2-eb66-4d9e-8081-9ebf708e2b17 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:08:03.717058 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c4470ba2-eb66-4d9e-8081-9ebf708e2b17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cc5317014d7424fb4052b517c7ae1c7', 'request_id': 'req-c4470ba2-eb66-4d9e-8081-9ebf708e2b17'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:08:03.724103 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c4470ba2-eb66-4d9e-8081-9ebf708e2b17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cc5317014d7424fb4052b517c7ae1c7', 'request_id': 'req-c4470ba2-eb66-4d9e-8081-9ebf708e2b17'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:08:03.729776 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c4470ba2-eb66-4d9e-8081-9ebf708e2b17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cc5317014d7424fb4052b517c7ae1c7', 'request_id': 'req-c4470ba2-eb66-4d9e-8081-9ebf708e2b17'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:08:03.741470 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c4470ba2-eb66-4d9e-8081-9ebf708e2b17 admin admin] The token cache is being invalidate because project 4cc5317014d7424fb4052b517c7ae1c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:08:03.741761 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c4470ba2-eb66-4d9e-8081-9ebf708e2b17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:08:03.742429 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c4470ba2-eb66-4d9e-8081-9ebf708e2b17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:08:03.743925 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 988/1961] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:08:03 2022] DELETE /identity/v3/projects/4cc5317014d7424fb4052b517c7ae1c7 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:08:03.749644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ebd7c-19d6-4d1b-9e94-c3d66420396e None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:03.749930 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ebd7c-19d6-4d1b-9e94-c3d66420396e None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:03.750203 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ebd7c-19d6-4d1b-9e94-c3d66420396e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:03.770022 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-2f3ebd7c-19d6-4d1b-9e94-c3d66420396e None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:08:03.771200 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2f3ebd7c-19d6-4d1b-9e94-c3d66420396e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:08:03.787387 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:08:03.831214 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 974/1962] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:08:03.839515 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ddd66-bb7b-42d0-9c95-543df63cc16c None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:03.839806 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ddd66-bb7b-42d0-9c95-543df63cc16c None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:03.840089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626ddd66-bb7b-42d0-9c95-543df63cc16c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:03.862048 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-626ddd66-bb7b-42d0-9c95-543df63cc16c None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:08:03.863304 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-626ddd66-bb7b-42d0-9c95-543df63cc16c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:08:03.874141 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 989/1963] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:08:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:08:03.878442 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ebd7c-19d6-4d1b-9e94-c3d66420396e None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:08:03.888725 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ebd7c-19d6-4d1b-9e94-c3d66420396e None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:08:03.893368 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05710770-2fc8-4d4e-b9ef-6419724a2fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:08:03.893967 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05710770-2fc8-4d4e-b9ef-6419724a2fb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:03.894274 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05710770-2fc8-4d4e-b9ef-6419724a2fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:03.894555 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05710770-2fc8-4d4e-b9ef-6419724a2fb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:03.895197 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05710770-2fc8-4d4e-b9ef-6419724a2fb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:08:03.895503 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05710770-2fc8-4d4e-b9ef-6419724a2fb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:08:03.897090 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05710770-2fc8-4d4e-b9ef-6419724a2fb2 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:08:03.908487 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 975/1964] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:08:03.913420 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ddd66-bb7b-42d0-9c95-543df63cc16c None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:08:03.923496 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626ddd66-bb7b-42d0-9c95-543df63cc16c None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:08:03.928826 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88149027-6c51-4b07-b92f-5e4990d59349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:08:03.929435 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88149027-6c51-4b07-b92f-5e4990d59349 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:03.929713 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88149027-6c51-4b07-b92f-5e4990d59349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:03.929983 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88149027-6c51-4b07-b92f-5e4990d59349 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:03.930661 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88149027-6c51-4b07-b92f-5e4990d59349 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:08:03.932561 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88149027-6c51-4b07-b92f-5e4990d59349 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:08:03.961033 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 990/1965] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:08:03 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:08:03.965183 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05710770-2fc8-4d4e-b9ef-6419724a2fb2 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:08:03.975824 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05710770-2fc8-4d4e-b9ef-6419724a2fb2 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:08:04.031297 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de5112-99c5-498c-8c57-7cffbf0a5298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:08:04.031914 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de5112-99c5-498c-8c57-7cffbf0a5298 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:04.032195 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de5112-99c5-498c-8c57-7cffbf0a5298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:04.032470 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de5112-99c5-498c-8c57-7cffbf0a5298 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:04.033125 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de5112-99c5-498c-8c57-7cffbf0a5298 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:08:04.035056 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de5112-99c5-498c-8c57-7cffbf0a5298 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:08:04.056788 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 976/1966] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:08:03 2022] POST /identity/v3/users => generated 282 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:08:04.060763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88149027-6c51-4b07-b92f-5e4990d59349 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:08:04.071501 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88149027-6c51-4b07-b92f-5e4990d59349 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:08:04.076175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0857e-6e25-4374-afaf-235c19682c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:08:04.076776 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0857e-6e25-4374-afaf-235c19682c95 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:04.077059 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0857e-6e25-4374-afaf-235c19682c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:04.077353 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe0857e-6e25-4374-afaf-235c19682c95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:04.077998 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe0857e-6e25-4374-afaf-235c19682c95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:08:04.079659 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe0857e-6e25-4374-afaf-235c19682c95 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:08:04.093709 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 991/1967] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:08:04.094954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de5112-99c5-498c-8c57-7cffbf0a5298 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:08:04.105637 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de5112-99c5-498c-8c57-7cffbf0a5298 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:08:04.110267 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875f5d70-6562-4424-b7b0-14f62c4f986a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:08:04.110868 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f5d70-6562-4424-b7b0-14f62c4f986a admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:04.111151 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f5d70-6562-4424-b7b0-14f62c4f986a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:04.111428 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875f5d70-6562-4424-b7b0-14f62c4f986a admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:04.112071 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875f5d70-6562-4424-b7b0-14f62c4f986a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:08:04.113652 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875f5d70-6562-4424-b7b0-14f62c4f986a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:08:04.124888 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 977/1968] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:08:04.129310 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0857e-6e25-4374-afaf-235c19682c95 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:08:04.139515 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe0857e-6e25-4374-afaf-235c19682c95 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:08:04.144275 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f15fb6-ccc7-4e09-9906-ec516bc0a5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:08:04.144957 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f15fb6-ccc7-4e09-9906-ec516bc0a5f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:04.145248 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f15fb6-ccc7-4e09-9906-ec516bc0a5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:04.145526 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f15fb6-ccc7-4e09-9906-ec516bc0a5f9 admin admin] PATH_INFO: `/v3/projects/b63c42249cc9429fafefc2e6dc7a25e5/users/04922aefa59f40a3a501309efe741003/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:04.161062 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f15fb6-ccc7-4e09-9906-ec516bc0a5f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b63c42249cc9429fafefc2e6dc7a25e5, user_id=04922aefa59f40a3a501309efe741003, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:08:04.164263 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f15fb6-ccc7-4e09-9906-ec516bc0a5f9 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:08:04.176908 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 992/1969] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:08:04 2022] PUT /identity/v3/projects/b63c42249cc9429fafefc2e6dc7a25e5/users/04922aefa59f40a3a501309efe741003/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:08:04.187725 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805fc949-6538-4de2-8c85-739a39eb55dd None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:04.188016 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805fc949-6538-4de2-8c85-739a39eb55dd None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:04.188296 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805fc949-6538-4de2-8c85-739a39eb55dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:04.215820 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-805fc949-6538-4de2-8c85-739a39eb55dd None None] MFA Rules not processed for user `04922aefa59f40a3a501309efe741003`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:08:04.217027 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-805fc949-6538-4de2-8c85-739a39eb55dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:08:04.280291 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 978/1970] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:08:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:08:04.291727 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f15fb6-ccc7-4e09-9906-ec516bc0a5f9 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:08:04.293447 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-c4f15fb6-ccc7-4e09-9906-ec516bc0a5f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:08:04.315803 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:08:04.354581 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f15fb6-ccc7-4e09-9906-ec516bc0a5f9 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:08:04.358502 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc75e6f-2f5d-4876-92e8-e3ef81203ac5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:08:04.359140 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc75e6f-2f5d-4876-92e8-e3ef81203ac5 service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:04.359417 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc75e6f-2f5d-4876-92e8-e3ef81203ac5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:04.359693 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc75e6f-2f5d-4876-92e8-e3ef81203ac5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:04.368332 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc75e6f-2f5d-4876-92e8-e3ef81203ac5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:08:04.369753 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc75e6f-2f5d-4876-92e8-e3ef81203ac5 service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:08:04.380254 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 993/1971] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:08:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:08:06.568132 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93e5f1-8db4-4a18-b817-dd3019c8b8a8 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:06.568629 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93e5f1-8db4-4a18-b817-dd3019c8b8a8 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:06.568726 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb93e5f1-8db4-4a18-b817-dd3019c8b8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:06.588264 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-bb93e5f1-8db4-4a18-b817-dd3019c8b8a8 None None] MFA Rules not processed for user `48be37e3311a4d6c968b135f8e5ee536`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:08:06.589466 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-bb93e5f1-8db4-4a18-b817-dd3019c8b8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:08:06.601605 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:08:06.644405 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 979/1972] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:08:06 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:08:13.111915 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc75e6f-2f5d-4876-92e8-e3ef81203ac5 service cinder] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:08:13.113411 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-cfc75e6f-2f5d-4876-92e8-e3ef81203ac5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:08:13.179302 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc75e6f-2f5d-4876-92e8-e3ef81203ac5 service cinder] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:08:13.183836 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215c1f7-beec-4800-a9c5-0c120ca84257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:08:13.184467 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215c1f7-beec-4800-a9c5-0c120ca84257 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:13.184747 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215c1f7-beec-4800-a9c5-0c120ca84257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:13.185020 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e215c1f7-beec-4800-a9c5-0c120ca84257 admin admin] PATH_INFO: `/v3/users/517d371acf6543d0ad7c098242a8b217` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:13.186307 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e215c1f7-beec-4800-a9c5-0c120ca84257 admin admin] RBAC: Authorizing `identity:delete_user(user_id=517d371acf6543d0ad7c098242a8b217)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:08:13.188259 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e215c1f7-beec-4800-a9c5-0c120ca84257 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:08:13.237478 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-e215c1f7-beec-4800-a9c5-0c120ca84257 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '517d371acf6543d0ad7c098242a8b217'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:08:13.252556 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-e215c1f7-beec-4800-a9c5-0c120ca84257 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '517d371acf6543d0ad7c098242a8b217'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:08:13.261040 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-e215c1f7-beec-4800-a9c5-0c120ca84257 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '517d371acf6543d0ad7c098242a8b217'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:08:13.270007 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-e215c1f7-beec-4800-a9c5-0c120ca84257 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '517d371acf6543d0ad7c098242a8b217'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:08:13.270720 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-e215c1f7-beec-4800-a9c5-0c120ca84257 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '517d371acf6543d0ad7c098242a8b217'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:08:13.278121 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 994/1973] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:08:13 2022] DELETE /identity/v3/users/517d371acf6543d0ad7c098242a8b217 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:08:13.282489 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b78cb-a79c-49a2-a65e-66692d076f2a None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:13.283348 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b78cb-a79c-49a2-a65e-66692d076f2a None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:13.283823 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b78cb-a79c-49a2-a65e-66692d076f2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:13.304713 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-4a7b78cb-a79c-49a2-a65e-66692d076f2a None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:08:13.305917 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4a7b78cb-a79c-49a2-a65e-66692d076f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:08:13.374964 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 980/1974] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:08:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:08:13.387633 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215c1f7-beec-4800-a9c5-0c120ca84257 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:08:13.390651 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-e215c1f7-beec-4800-a9c5-0c120ca84257 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:08:13.450788 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:08:13.469438 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e215c1f7-beec-4800-a9c5-0c120ca84257 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:08:13.474583 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e04a17e-df5b-4216-81d8-8aafad732a50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:08:13.475340 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04a17e-df5b-4216-81d8-8aafad732a50 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:13.479499 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04a17e-df5b-4216-81d8-8aafad732a50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:13.479800 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e04a17e-df5b-4216-81d8-8aafad732a50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:13.491867 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e04a17e-df5b-4216-81d8-8aafad732a50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:08:13.493605 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e04a17e-df5b-4216-81d8-8aafad732a50 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:08:13.508613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 995/1975] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:08:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:08:13.892893 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b78cb-a79c-49a2-a65e-66692d076f2a None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:08:13.894385 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4a7b78cb-a79c-49a2-a65e-66692d076f2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:08:13.910102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b78cb-a79c-49a2-a65e-66692d076f2a None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:08:13.915784 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ffcdc2-da41-4736-b58b-4af731f4ee15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:08:13.917582 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ffcdc2-da41-4736-b58b-4af731f4ee15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:13.918072 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ffcdc2-da41-4736-b58b-4af731f4ee15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:13.918314 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ffcdc2-da41-4736-b58b-4af731f4ee15 admin admin] PATH_INFO: `/v3/projects/884950c9900747699b76fd43700ca8a9` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:13.919652 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ffcdc2-da41-4736-b58b-4af731f4ee15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=884950c9900747699b76fd43700ca8a9)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:08:13.921513 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ffcdc2-da41-4736-b58b-4af731f4ee15 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:08:13.972283 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-00ffcdc2-da41-4736-b58b-4af731f4ee15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '884950c9900747699b76fd43700ca8a9', 'request_id': 'req-00ffcdc2-da41-4736-b58b-4af731f4ee15'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:08:13.981748 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-00ffcdc2-da41-4736-b58b-4af731f4ee15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '884950c9900747699b76fd43700ca8a9', 'request_id': 'req-00ffcdc2-da41-4736-b58b-4af731f4ee15'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:08:13.987627 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-00ffcdc2-da41-4736-b58b-4af731f4ee15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '884950c9900747699b76fd43700ca8a9', 'request_id': 'req-00ffcdc2-da41-4736-b58b-4af731f4ee15'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:08:14.004069 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-00ffcdc2-da41-4736-b58b-4af731f4ee15 admin admin] The token cache is being invalidate because project 884950c9900747699b76fd43700ca8a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:08:14.004361 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-00ffcdc2-da41-4736-b58b-4af731f4ee15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:08:14.005024 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-00ffcdc2-da41-4736-b58b-4af731f4ee15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:08:14.006580 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 981/1976] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:08:13 2022] DELETE /identity/v3/projects/884950c9900747699b76fd43700ca8a9 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:08:14.015670 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de066f9d-02f2-4462-913c-a036ccd34694 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:14.016280 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de066f9d-02f2-4462-913c-a036ccd34694 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:14.016580 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de066f9d-02f2-4462-913c-a036ccd34694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:14.036136 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-de066f9d-02f2-4462-913c-a036ccd34694 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:08:14.037900 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-de066f9d-02f2-4462-913c-a036ccd34694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:08:14.096743 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 996/1977] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:08:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:08:14.106084 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48ad58-8008-4abb-b737-dc1ac1fbf2d4 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:14.106409 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48ad58-8008-4abb-b737-dc1ac1fbf2d4 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:14.106723 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d48ad58-8008-4abb-b737-dc1ac1fbf2d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:14.127196 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-3d48ad58-8008-4abb-b737-dc1ac1fbf2d4 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:08:14.128403 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3d48ad58-8008-4abb-b737-dc1ac1fbf2d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:08:14.136146 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 982/1978] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:08:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:08:14.140242 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de066f9d-02f2-4462-913c-a036ccd34694 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:08:14.150396 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de066f9d-02f2-4462-913c-a036ccd34694 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:08:14.155104 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf98204-e2f9-4661-b44e-38a39b3627cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:08:14.155716 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf98204-e2f9-4661-b44e-38a39b3627cf admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:14.155998 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf98204-e2f9-4661-b44e-38a39b3627cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:14.156270 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf98204-e2f9-4661-b44e-38a39b3627cf admin admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:14.156952 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf98204-e2f9-4661-b44e-38a39b3627cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:08:14.157255 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf98204-e2f9-4661-b44e-38a39b3627cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:08:14.158882 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf98204-e2f9-4661-b44e-38a39b3627cf admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:08:14.171460 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 997/1979] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:08:14.175664 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d48ad58-8008-4abb-b737-dc1ac1fbf2d4 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:08:14.185381 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d48ad58-8008-4abb-b737-dc1ac1fbf2d4 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:08:14.190188 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b5ca85-9a18-4a38-afc5-898cc8b613fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:08:14.190833 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5ca85-9a18-4a38-afc5-898cc8b613fc admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:14.191113 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5ca85-9a18-4a38-afc5-898cc8b613fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:14.191390 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5ca85-9a18-4a38-afc5-898cc8b613fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:14.192024 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b5ca85-9a18-4a38-afc5-898cc8b613fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:08:14.193957 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b5ca85-9a18-4a38-afc5-898cc8b613fc admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:08:14.225756 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 983/1980] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:08:14 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:08:14.226745 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf98204-e2f9-4661-b44e-38a39b3627cf admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:08:14.236818 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf98204-e2f9-4661-b44e-38a39b3627cf admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:08:14.301604 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd708a-af94-4249-8c23-1aaac2f170fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:08:14.302365 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd708a-af94-4249-8c23-1aaac2f170fc admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:14.302642 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd708a-af94-4249-8c23-1aaac2f170fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:14.302937 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddd708a-af94-4249-8c23-1aaac2f170fc admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:14.303653 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd708a-af94-4249-8c23-1aaac2f170fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:08:14.305597 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddd708a-af94-4249-8c23-1aaac2f170fc admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:08:14.328966 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 998/1981] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:08:14 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:08:14.334808 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b5ca85-9a18-4a38-afc5-898cc8b613fc admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:08:14.345790 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b5ca85-9a18-4a38-afc5-898cc8b613fc admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:08:14.351189 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4329a9-093d-422d-9ec8-cfc235b92320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:08:14.351825 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4329a9-093d-422d-9ec8-cfc235b92320 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:14.352095 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4329a9-093d-422d-9ec8-cfc235b92320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:14.352371 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4329a9-093d-422d-9ec8-cfc235b92320 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:14.353023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4329a9-093d-422d-9ec8-cfc235b92320 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:08:14.354681 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4329a9-093d-422d-9ec8-cfc235b92320 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:08:14.365721 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 984/1982] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 20:08:14.369796 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd708a-af94-4249-8c23-1aaac2f170fc admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:08:14.379799 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddd708a-af94-4249-8c23-1aaac2f170fc admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:08:14.384470 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efcaae1-1acb-4323-bfaa-f8de83641757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:08:14.385123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcaae1-1acb-4323-bfaa-f8de83641757 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:14.385486 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcaae1-1acb-4323-bfaa-f8de83641757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:14.385753 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcaae1-1acb-4323-bfaa-f8de83641757 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:14.386440 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efcaae1-1acb-4323-bfaa-f8de83641757 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:08:14.388037 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efcaae1-1acb-4323-bfaa-f8de83641757 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:08:14.399435 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 999/1983] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:08:14.403504 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4329a9-093d-422d-9ec8-cfc235b92320 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:08:14.413951 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4329a9-093d-422d-9ec8-cfc235b92320 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:08:14.419478 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c33195-ea22-43f1-bd03-3b1f1ac8c0b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:08:14.420196 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c33195-ea22-43f1-bd03-3b1f1ac8c0b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:14.420479 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c33195-ea22-43f1-bd03-3b1f1ac8c0b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:14.420779 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c33195-ea22-43f1-bd03-3b1f1ac8c0b1 admin admin] PATH_INFO: `/v3/projects/ae31769b5e4c4b2e9680f86cd820784f/users/c9cf226192c94300bafdd3bac13ab8bc/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:14.423253 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:08:14.438175 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c33195-ea22-43f1-bd03-3b1f1ac8c0b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae31769b5e4c4b2e9680f86cd820784f, user_id=c9cf226192c94300bafdd3bac13ab8bc, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:08:14.440955 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c33195-ea22-43f1-bd03-3b1f1ac8c0b1 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:08:14.454212 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 985/1984] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:08:14 2022] PUT /identity/v3/projects/ae31769b5e4c4b2e9680f86cd820784f/users/c9cf226192c94300bafdd3bac13ab8bc/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:08:14.464966 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28481957-be9f-48a6-a176-a3c457460106 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:14.465252 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28481957-be9f-48a6-a176-a3c457460106 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:14.465578 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28481957-be9f-48a6-a176-a3c457460106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:14.492796 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-28481957-be9f-48a6-a176-a3c457460106 None None] MFA Rules not processed for user `c9cf226192c94300bafdd3bac13ab8bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:08:14.493965 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-28481957-be9f-48a6-a176-a3c457460106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:08:14.535735 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:08:14.561496 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1000/1985] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:08:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:08:14.572910 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c33195-ea22-43f1-bd03-3b1f1ac8c0b1 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:08:14.574744 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-72c33195-ea22-43f1-bd03-3b1f1ac8c0b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:08:14.633250 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c33195-ea22-43f1-bd03-3b1f1ac8c0b1 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:08:14.637107 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec423d77-eed8-447c-bd25-27102acbf66d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:08:14.638868 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec423d77-eed8-447c-bd25-27102acbf66d service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:14.639155 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec423d77-eed8-447c-bd25-27102acbf66d service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:14.639439 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec423d77-eed8-447c-bd25-27102acbf66d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:14.648977 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec423d77-eed8-447c-bd25-27102acbf66d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:08:14.650442 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec423d77-eed8-447c-bd25-27102acbf66d service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:08:14.659493 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 986/1986] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:08:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:08:20.254270 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92063deb-d97a-49ab-bc9b-d8f1f05a723d None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:20.254270 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92063deb-d97a-49ab-bc9b-d8f1f05a723d None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:20.254270 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92063deb-d97a-49ab-bc9b-d8f1f05a723d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:20.270408 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-92063deb-d97a-49ab-bc9b-d8f1f05a723d None None] MFA Rules not processed for user `c9cf226192c94300bafdd3bac13ab8bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:08:20.272158 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-92063deb-d97a-49ab-bc9b-d8f1f05a723d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:08:20.288553 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1001/1987] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:08:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:08:21.724282 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec423d77-eed8-447c-bd25-27102acbf66d service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:08:21.725796 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ec423d77-eed8-447c-bd25-27102acbf66d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:08:21.793766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec423d77-eed8-447c-bd25-27102acbf66d service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:08:21.797620 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5c7caf-fd83-4628-b16a-48af412c992b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:08:21.798228 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c7caf-fd83-4628-b16a-48af412c992b service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:21.798572 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c7caf-fd83-4628-b16a-48af412c992b service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:21.798843 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c7caf-fd83-4628-b16a-48af412c992b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:21.800762 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-da5c7caf-fd83-4628-b16a-48af412c992b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:08:21.901388 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5c7caf-fd83-4628-b16a-48af412c992b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:08:21.904100 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5c7caf-fd83-4628-b16a-48af412c992b service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:08:21.920456 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 987/1988] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:08:21 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:08:22.963488 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92063deb-d97a-49ab-bc9b-d8f1f05a723d None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:08:22.964977 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-92063deb-d97a-49ab-bc9b-d8f1f05a723d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:08:23.069214 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92063deb-d97a-49ab-bc9b-d8f1f05a723d None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:08:23.075254 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27117e73-19e8-43a0-8d1d-b12504144945 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312489c49a304ca9aacebd9843a4b65f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:08:23.076488 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27117e73-19e8-43a0-8d1d-b12504144945 service placement] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:23.076889 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27117e73-19e8-43a0-8d1d-b12504144945 service placement] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:23.077283 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27117e73-19e8-43a0-8d1d-b12504144945 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:23.079287 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-27117e73-19e8-43a0-8d1d-b12504144945 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:08:23.093868 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27117e73-19e8-43a0-8d1d-b12504144945 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:08:23.096392 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27117e73-19e8-43a0-8d1d-b12504144945 service placement] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:08:23.111399 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1002/1989] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:08:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:08:31.914888 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=83,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:08:33.104283 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=44 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:08:34.369282 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda9b9e-a4b8-4aa4-aaa1-179f11634f11 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:34.369807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda9b9e-a4b8-4aa4-aaa1-179f11634f11 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:34.369925 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fda9b9e-a4b8-4aa4-aaa1-179f11634f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:34.389452 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-2fda9b9e-a4b8-4aa4-aaa1-179f11634f11 None None] MFA Rules not processed for user `4b6a101d23df4b62ba7e0c35d28d2a3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:08:34.390691 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2fda9b9e-a4b8-4aa4-aaa1-179f11634f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:08:34.455645 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 988/1990] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:08:34 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:08:43.968981 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd79986-0f5a-4de8-8cea-3cd1465d244c None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:43.969437 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd79986-0f5a-4de8-8cea-3cd1465d244c None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:43.969658 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd79986-0f5a-4de8-8cea-3cd1465d244c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:43.991284 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-4cd79986-0f5a-4de8-8cea-3cd1465d244c None None] MFA Rules not processed for user `ff5cfe4d31614e2db9059c69116297cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:08:43.992506 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-4cd79986-0f5a-4de8-8cea-3cd1465d244c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:08:44.054325 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1003/1991] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Oct 23 20:08:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:08:44.093805 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda9b9e-a4b8-4aa4-aaa1-179f11634f11 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:08:44.095190 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2fda9b9e-a4b8-4aa4-aaa1-179f11634f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:08:44.167178 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fda9b9e-a4b8-4aa4-aaa1-179f11634f11 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:08:44.172339 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cbb221-359b-4cce-bbe0-a691aa96567c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15f603d1e7f54751aa79317e2ea00a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:08:44.172730 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cbb221-359b-4cce-bbe0-a691aa96567c service swift] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:44.173001 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cbb221-359b-4cce-bbe0-a691aa96567c service swift] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:44.173271 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cbb221-359b-4cce-bbe0-a691aa96567c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:44.181771 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cbb221-359b-4cce-bbe0-a691aa96567c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:08:44.183696 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cbb221-359b-4cce-bbe0-a691aa96567c service swift] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:08:44.192638 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 989/1992] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Oct 23 20:08:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 20:08:54.047273 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=25 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:08:54.189221 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=57 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:08:56.508611 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62074935-0bf2-44a2-9c8d-09e40c78fa20 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:56.509586 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62074935-0bf2-44a2-9c8d-09e40c78fa20 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:56.510020 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62074935-0bf2-44a2-9c8d-09e40c78fa20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:56.543948 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-62074935-0bf2-44a2-9c8d-09e40c78fa20 None None] MFA Rules not processed for user `ff5cfe4d31614e2db9059c69116297cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:08:56.545709 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-62074935-0bf2-44a2-9c8d-09e40c78fa20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:08:56.558450 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1004/1993] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Oct 23 20:08:56 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:08:56.573139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cbb221-359b-4cce-bbe0-a691aa96567c service swift] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:08:56.598087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cbb221-359b-4cce-bbe0-a691aa96567c service swift] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:08:56.622183 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af0bdf-dfd5-42da-9a4a-2b0aec3053e4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15f603d1e7f54751aa79317e2ea00a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:08:56.622844 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af0bdf-dfd5-42da-9a4a-2b0aec3053e4 service swift] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:56.623131 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af0bdf-dfd5-42da-9a4a-2b0aec3053e4 service swift] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:56.623412 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74af0bdf-dfd5-42da-9a4a-2b0aec3053e4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:56.641120 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af0bdf-dfd5-42da-9a4a-2b0aec3053e4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:08:56.642574 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74af0bdf-dfd5-42da-9a4a-2b0aec3053e4 service swift] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:08:56.665728 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 990/1994] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Oct 23 20:08:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 20:08:57.538990 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de81a9-b56d-403f-8dc6-fa0b7a933dc1 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:57.539570 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de81a9-b56d-403f-8dc6-fa0b7a933dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:57.539757 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40de81a9-b56d-403f-8dc6-fa0b7a933dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:57.560683 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-40de81a9-b56d-403f-8dc6-fa0b7a933dc1 None None] MFA Rules not processed for user `ff5cfe4d31614e2db9059c69116297cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:08:57.562808 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-40de81a9-b56d-403f-8dc6-fa0b7a933dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:08:57.572111 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1005/1995] 10.222.0.13 () {58 vars in 967 bytes} [Sun Oct 23 20:08:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:08:57.586000 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af0bdf-dfd5-42da-9a4a-2b0aec3053e4 service swift] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:08:57.597204 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74af0bdf-dfd5-42da-9a4a-2b0aec3053e4 service swift] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:08:57.603074 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c0163-2b0c-4f5d-8d8c-41ce45aabda3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15f603d1e7f54751aa79317e2ea00a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:08:57.603933 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c0163-2b0c-4f5d-8d8c-41ce45aabda3 service swift] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:57.604456 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c0163-2b0c-4f5d-8d8c-41ce45aabda3 service swift] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:57.604913 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c0163-2b0c-4f5d-8d8c-41ce45aabda3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:57.615675 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1c0163-2b0c-4f5d-8d8c-41ce45aabda3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:08:57.617676 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1c0163-2b0c-4f5d-8d8c-41ce45aabda3 service swift] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:08:57.627693 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 991/1996] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Oct 23 20:08:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 20:08:57.942505 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc13422-6f70-4151-8c4e-4a3c87283541 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:57.942877 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc13422-6f70-4151-8c4e-4a3c87283541 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:57.943155 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc13422-6f70-4151-8c4e-4a3c87283541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:57.971621 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-3fc13422-6f70-4151-8c4e-4a3c87283541 None None] MFA Rules not processed for user `ff5cfe4d31614e2db9059c69116297cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:08:57.973022 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-3fc13422-6f70-4151-8c4e-4a3c87283541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:08:57.981693 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1006/1997] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Oct 23 20:08:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:08:57.995396 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c0163-2b0c-4f5d-8d8c-41ce45aabda3 service swift] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:08:58.007663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c0163-2b0c-4f5d-8d8c-41ce45aabda3 service swift] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:08:58.013291 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7c6823-8b6c-4090-9cfb-bdcd219ca9d8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15f603d1e7f54751aa79317e2ea00a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:08:58.013909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c6823-8b6c-4090-9cfb-bdcd219ca9d8 service swift] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:58.014399 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c6823-8b6c-4090-9cfb-bdcd219ca9d8 service swift] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:58.014873 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7c6823-8b6c-4090-9cfb-bdcd219ca9d8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:58.025038 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7c6823-8b6c-4090-9cfb-bdcd219ca9d8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:08:58.027314 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7c6823-8b6c-4090-9cfb-bdcd219ca9d8 service swift] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:08:58.045703 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 992/1998] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Oct 23 20:08:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 20:08:58.480107 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bce6a8-10b0-446a-befa-24f51803a76c None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:58.480700 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bce6a8-10b0-446a-befa-24f51803a76c None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:58.481153 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bce6a8-10b0-446a-befa-24f51803a76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:58.502361 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-b4bce6a8-10b0-446a-befa-24f51803a76c None None] MFA Rules not processed for user `ff5cfe4d31614e2db9059c69116297cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:08:58.503723 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-b4bce6a8-10b0-446a-befa-24f51803a76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:08:58.512004 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1007/1999] 10.222.0.13 () {58 vars in 967 bytes} [Sun Oct 23 20:08:58 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:08:58.527443 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7c6823-8b6c-4090-9cfb-bdcd219ca9d8 service swift] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:08:58.542161 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7c6823-8b6c-4090-9cfb-bdcd219ca9d8 service swift] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:08:58.548994 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2ba925-0504-475c-bf52-cdf8697c8ea0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15f603d1e7f54751aa79317e2ea00a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:08:58.549817 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ba925-0504-475c-bf52-cdf8697c8ea0 service swift] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:08:58.550295 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ba925-0504-475c-bf52-cdf8697c8ea0 service swift] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:08:58.550582 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2ba925-0504-475c-bf52-cdf8697c8ea0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:08:58.559720 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2ba925-0504-475c-bf52-cdf8697c8ea0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:08:58.561371 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2ba925-0504-475c-bf52-cdf8697c8ea0 service swift] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:08:58.571384 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 993/2000] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Oct 23 20:08:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 23 20:09:00.345496 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bce6a8-10b0-446a-befa-24f51803a76c None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:09:00.347131 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-b4bce6a8-10b0-446a-befa-24f51803a76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:09:00.429935 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bce6a8-10b0-446a-befa-24f51803a76c None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:09:00.434568 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5ddf1f-cbae-4b39-880c-ac518134d5c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:09:00.435207 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5ddf1f-cbae-4b39-880c-ac518134d5c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:00.435489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5ddf1f-cbae-4b39-880c-ac518134d5c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:00.435764 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5ddf1f-cbae-4b39-880c-ac518134d5c2 admin admin] PATH_INFO: `/v3/users/04922aefa59f40a3a501309efe741003` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:00.437035 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5ddf1f-cbae-4b39-880c-ac518134d5c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04922aefa59f40a3a501309efe741003)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:09:00.439001 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5ddf1f-cbae-4b39-880c-ac518134d5c2 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:09:00.488195 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-2e5ddf1f-cbae-4b39-880c-ac518134d5c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04922aefa59f40a3a501309efe741003'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:00.503891 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-2e5ddf1f-cbae-4b39-880c-ac518134d5c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04922aefa59f40a3a501309efe741003'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:00.511490 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-2e5ddf1f-cbae-4b39-880c-ac518134d5c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04922aefa59f40a3a501309efe741003'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:00.521712 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-2e5ddf1f-cbae-4b39-880c-ac518134d5c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04922aefa59f40a3a501309efe741003'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:00.522556 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-2e5ddf1f-cbae-4b39-880c-ac518134d5c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04922aefa59f40a3a501309efe741003'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:00.528851 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1008/2001] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:09:00 2022] DELETE /identity/v3/users/04922aefa59f40a3a501309efe741003 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:09:00.533400 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec35fb2-15ee-4056-876f-85bc809806be None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:00.533947 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec35fb2-15ee-4056-876f-85bc809806be None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:00.534141 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec35fb2-15ee-4056-876f-85bc809806be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:00.555084 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-8ec35fb2-15ee-4056-876f-85bc809806be None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:09:00.556287 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-8ec35fb2-15ee-4056-876f-85bc809806be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:09:00.615736 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 994/2002] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:09:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:09:00.625448 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5ddf1f-cbae-4b39-880c-ac518134d5c2 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:09:00.626895 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-2e5ddf1f-cbae-4b39-880c-ac518134d5c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:09:00.683836 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5ddf1f-cbae-4b39-880c-ac518134d5c2 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:09:00.687559 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4786457-80a9-42b2-9634-dd17036a4760 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:09:00.688159 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4786457-80a9-42b2-9634-dd17036a4760 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:00.688437 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4786457-80a9-42b2-9634-dd17036a4760 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:00.688708 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4786457-80a9-42b2-9634-dd17036a4760 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:00.696662 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:09:00.698936 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4786457-80a9-42b2-9634-dd17036a4760 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:09:00.700659 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4786457-80a9-42b2-9634-dd17036a4760 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:09:00.712575 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1009/2003] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:09:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:09:00.974071 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec35fb2-15ee-4056-876f-85bc809806be None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:09:00.975561 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-8ec35fb2-15ee-4056-876f-85bc809806be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:09:00.976884 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4786457-80a9-42b2-9634-dd17036a4760 service neutron] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:09:00.978398 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-d4786457-80a9-42b2-9634-dd17036a4760 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:09:00.989791 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec35fb2-15ee-4056-876f-85bc809806be None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:09:00.991998 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4786457-80a9-42b2-9634-dd17036a4760 service neutron] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:09:00.995629 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc1d575-4ad7-4c65-97a8-2d9aa2d7f5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:09:00.996464 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc1d575-4ad7-4c65-97a8-2d9aa2d7f5c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:00.996917 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc1d575-4ad7-4c65-97a8-2d9aa2d7f5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:00.997057 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b412748-f890-46c1-81b0-893540ab501e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:09:00.997188 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc1d575-4ad7-4c65-97a8-2d9aa2d7f5c8 admin admin] PATH_INFO: `/v3/users/c9cf226192c94300bafdd3bac13ab8bc` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:00.997741 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b412748-f890-46c1-81b0-893540ab501e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:00.998077 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b412748-f890-46c1-81b0-893540ab501e admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:00.998452 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b412748-f890-46c1-81b0-893540ab501e admin admin] PATH_INFO: `/v3/projects/b63c42249cc9429fafefc2e6dc7a25e5` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:00.998837 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc1d575-4ad7-4c65-97a8-2d9aa2d7f5c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9cf226192c94300bafdd3bac13ab8bc)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:09:00.999776 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b412748-f890-46c1-81b0-893540ab501e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b63c42249cc9429fafefc2e6dc7a25e5)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:09:01.000776 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc1d575-4ad7-4c65-97a8-2d9aa2d7f5c8 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:09:01.001794 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b412748-f890-46c1-81b0-893540ab501e admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:09:01.051837 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9dc1d575-4ad7-4c65-97a8-2d9aa2d7f5c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9cf226192c94300bafdd3bac13ab8bc'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:01.053235 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-3b412748-f890-46c1-81b0-893540ab501e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b63c42249cc9429fafefc2e6dc7a25e5', 'request_id': 'req-3b412748-f890-46c1-81b0-893540ab501e'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:01.061545 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-3b412748-f890-46c1-81b0-893540ab501e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b63c42249cc9429fafefc2e6dc7a25e5', 'request_id': 'req-3b412748-f890-46c1-81b0-893540ab501e'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:01.066540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9dc1d575-4ad7-4c65-97a8-2d9aa2d7f5c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9cf226192c94300bafdd3bac13ab8bc'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:01.068348 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-3b412748-f890-46c1-81b0-893540ab501e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b63c42249cc9429fafefc2e6dc7a25e5', 'request_id': 'req-3b412748-f890-46c1-81b0-893540ab501e'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:01.075139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9dc1d575-4ad7-4c65-97a8-2d9aa2d7f5c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9cf226192c94300bafdd3bac13ab8bc'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:01.075866 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9dc1d575-4ad7-4c65-97a8-2d9aa2d7f5c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9cf226192c94300bafdd3bac13ab8bc'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:01.080336 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-3b412748-f890-46c1-81b0-893540ab501e admin admin] The token cache is being invalidate because project b63c42249cc9429fafefc2e6dc7a25e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:09:01.080628 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-3b412748-f890-46c1-81b0-893540ab501e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:01.081269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-3b412748-f890-46c1-81b0-893540ab501e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:01.083054 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1010/2004] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:09:00 2022] DELETE /identity/v3/projects/b63c42249cc9429fafefc2e6dc7a25e5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:09:01.083189 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9dc1d575-4ad7-4c65-97a8-2d9aa2d7f5c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9cf226192c94300bafdd3bac13ab8bc'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:01.087671 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a638898-037f-4a65-a764-c8b7817e39ee None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:01.088033 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a638898-037f-4a65-a764-c8b7817e39ee None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:01.088380 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a638898-037f-4a65-a764-c8b7817e39ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:01.089105 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 995/2005] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:09:00 2022] DELETE /identity/v3/users/c9cf226192c94300bafdd3bac13ab8bc => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:09:01.093196 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881815d1-db37-4320-a6f5-504f351fa2dc None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:01.093484 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881815d1-db37-4320-a6f5-504f351fa2dc None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:01.093761 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881815d1-db37-4320-a6f5-504f351fa2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:01.108203 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-3a638898-037f-4a65-a764-c8b7817e39ee None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:09:01.109506 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-3a638898-037f-4a65-a764-c8b7817e39ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:09:01.115048 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-881815d1-db37-4320-a6f5-504f351fa2dc None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:09:01.116262 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-881815d1-db37-4320-a6f5-504f351fa2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:09:01.129342 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:09:01.172380 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1011/2006] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:09:01.177639 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 996/2007] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:09:01.184738 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157ed7e-6c9c-4188-ac66-26721f755dbb None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:01.185103 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157ed7e-6c9c-4188-ac66-26721f755dbb None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:01.185512 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157ed7e-6c9c-4188-ac66-26721f755dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:01.187223 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881815d1-db37-4320-a6f5-504f351fa2dc None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:09:01.188583 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-881815d1-db37-4320-a6f5-504f351fa2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:09:01.206441 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-1157ed7e-6c9c-4188-ac66-26721f755dbb None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:09:01.207602 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-1157ed7e-6c9c-4188-ac66-26721f755dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:09:01.215577 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1012/2008] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:09:01.219600 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1157ed7e-6c9c-4188-ac66-26721f755dbb None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:09:01.229017 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1157ed7e-6c9c-4188-ac66-26721f755dbb None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:09:01.233643 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def768c-8bb7-4262-907f-8cf902b1a121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:09:01.234274 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def768c-8bb7-4262-907f-8cf902b1a121 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:01.234558 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def768c-8bb7-4262-907f-8cf902b1a121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:01.234834 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6def768c-8bb7-4262-907f-8cf902b1a121 admin admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:01.235478 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6def768c-8bb7-4262-907f-8cf902b1a121 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:09:01.235783 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6def768c-8bb7-4262-907f-8cf902b1a121 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:09:01.237367 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6def768c-8bb7-4262-907f-8cf902b1a121 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:09:01.252957 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1013/2009] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:09:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 20:09:01.253137 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-881815d1-db37-4320-a6f5-504f351fa2dc None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:09:01.253888 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def768c-8bb7-4262-907f-8cf902b1a121 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:09:01.254206 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e662a4-82c0-44bd-8620-c039efe9da42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:09:01.254939 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e662a4-82c0-44bd-8620-c039efe9da42 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:01.255215 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e662a4-82c0-44bd-8620-c039efe9da42 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:01.255485 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e662a4-82c0-44bd-8620-c039efe9da42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:01.263369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6def768c-8bb7-4262-907f-8cf902b1a121 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:09:01.264616 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e662a4-82c0-44bd-8620-c039efe9da42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:09:01.266035 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e662a4-82c0-44bd-8620-c039efe9da42 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:09:01.268166 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9d1100-a6b7-4d39-92a1-1954c9da30a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:09:01.268745 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d1100-a6b7-4d39-92a1-1954c9da30a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:01.269027 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d1100-a6b7-4d39-92a1-1954c9da30a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:01.269309 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9d1100-a6b7-4d39-92a1-1954c9da30a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:01.270480 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9d1100-a6b7-4d39-92a1-1954c9da30a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:09:01.272368 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9d1100-a6b7-4d39-92a1-1954c9da30a2 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:09:01.276002 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 997/2010] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:09:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 20:09:01.303498 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1014/2011] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:09:01 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:09:01.304559 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e662a4-82c0-44bd-8620-c039efe9da42 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:09:01.314132 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e662a4-82c0-44bd-8620-c039efe9da42 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:09:01.372480 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e636af5-f321-4083-8333-5054016e915d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:09:01.373096 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e636af5-f321-4083-8333-5054016e915d admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:01.373380 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e636af5-f321-4083-8333-5054016e915d admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:01.373654 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e636af5-f321-4083-8333-5054016e915d admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:01.374323 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e636af5-f321-4083-8333-5054016e915d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:09:01.376225 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e636af5-f321-4083-8333-5054016e915d admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:09:01.398937 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 998/2012] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:09:01 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:09:01.403332 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9d1100-a6b7-4d39-92a1-1954c9da30a2 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:09:01.413867 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9d1100-a6b7-4d39-92a1-1954c9da30a2 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:09:01.418614 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd598805-1be7-4e0a-b182-e31e247a23b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:09:01.419251 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd598805-1be7-4e0a-b182-e31e247a23b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:01.419529 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd598805-1be7-4e0a-b182-e31e247a23b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:01.419798 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd598805-1be7-4e0a-b182-e31e247a23b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:01.420466 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd598805-1be7-4e0a-b182-e31e247a23b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:09:01.422057 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd598805-1be7-4e0a-b182-e31e247a23b9 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:09:01.432402 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1015/2013] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:09:01.436844 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e636af5-f321-4083-8333-5054016e915d admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:09:01.448706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e636af5-f321-4083-8333-5054016e915d admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:09:01.452163 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd598805-1be7-4e0a-b182-e31e247a23b9 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:09:01.453891 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af97d01-d1c9-4932-8ad9-3bfdb5460cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:09:01.454432 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-bd598805-1be7-4e0a-b182-e31e247a23b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:09:01.454571 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af97d01-d1c9-4932-8ad9-3bfdb5460cac admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:01.454916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af97d01-d1c9-4932-8ad9-3bfdb5460cac admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:01.455112 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af97d01-d1c9-4932-8ad9-3bfdb5460cac admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:01.455766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af97d01-d1c9-4932-8ad9-3bfdb5460cac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:09:01.458112 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af97d01-d1c9-4932-8ad9-3bfdb5460cac admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:09:01.474289 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 999/2014] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:09:01.478039 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af97d01-d1c9-4932-8ad9-3bfdb5460cac admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:09:01.481925 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd598805-1be7-4e0a-b182-e31e247a23b9 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:09:01.486743 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70215d2f-df67-4cb0-8495-e7ad54257309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:09:01.487382 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70215d2f-df67-4cb0-8495-e7ad54257309 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:01.487663 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70215d2f-df67-4cb0-8495-e7ad54257309 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:01.487938 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70215d2f-df67-4cb0-8495-e7ad54257309 admin admin] PATH_INFO: `/v3/projects/ae31769b5e4c4b2e9680f86cd820784f` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:01.489007 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af97d01-d1c9-4932-8ad9-3bfdb5460cac admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:09:01.489123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70215d2f-df67-4cb0-8495-e7ad54257309 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae31769b5e4c4b2e9680f86cd820784f)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:09:01.491048 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70215d2f-df67-4cb0-8495-e7ad54257309 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:09:01.493951 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b10c46-b9b9-4f6d-890c-ad24c61a5dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:09:01.494633 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b10c46-b9b9-4f6d-890c-ad24c61a5dcc admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:01.494915 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b10c46-b9b9-4f6d-890c-ad24c61a5dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:01.495192 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b10c46-b9b9-4f6d-890c-ad24c61a5dcc admin admin] PATH_INFO: `/v3/projects/f84aaa230cf14fd2b47b2076e43fcb73/users/21fac7ecf4f2435483df150292a0052b/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:01.510729 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b10c46-b9b9-4f6d-890c-ad24c61a5dcc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f84aaa230cf14fd2b47b2076e43fcb73, user_id=21fac7ecf4f2435483df150292a0052b, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:09:01.513534 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b10c46-b9b9-4f6d-890c-ad24c61a5dcc admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:09:01.514524 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:09:01.528684 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1000/2015] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:09:01 2022] PUT /identity/v3/projects/f84aaa230cf14fd2b47b2076e43fcb73/users/21fac7ecf4f2435483df150292a0052b/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:09:01.533442 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e3cb87-2be9-4f1e-aa4a-a26419f06c9c None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:01.533996 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e3cb87-2be9-4f1e-aa4a-a26419f06c9c None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:01.533996 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e3cb87-2be9-4f1e-aa4a-a26419f06c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:01.541786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-70215d2f-df67-4cb0-8495-e7ad54257309 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae31769b5e4c4b2e9680f86cd820784f', 'request_id': 'req-70215d2f-df67-4cb0-8495-e7ad54257309'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:01.549805 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-70215d2f-df67-4cb0-8495-e7ad54257309 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae31769b5e4c4b2e9680f86cd820784f', 'request_id': 'req-70215d2f-df67-4cb0-8495-e7ad54257309'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:01.553040 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-01e3cb87-2be9-4f1e-aa4a-a26419f06c9c None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:09:01.554261 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-01e3cb87-2be9-4f1e-aa4a-a26419f06c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:09:01.555543 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-70215d2f-df67-4cb0-8495-e7ad54257309 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae31769b5e4c4b2e9680f86cd820784f', 'request_id': 'req-70215d2f-df67-4cb0-8495-e7ad54257309'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:01.566336 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:09:01.570007 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-70215d2f-df67-4cb0-8495-e7ad54257309 admin admin] The token cache is being invalidate because project ae31769b5e4c4b2e9680f86cd820784f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:09:01.570330 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-70215d2f-df67-4cb0-8495-e7ad54257309 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:01.571261 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-70215d2f-df67-4cb0-8495-e7ad54257309 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:01.582363 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1016/2016] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:09:01 2022] DELETE /identity/v3/projects/ae31769b5e4c4b2e9680f86cd820784f => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:09:01.653296 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1001/2017] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:09:01.667876 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70215d2f-df67-4cb0-8495-e7ad54257309 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:09:01.669415 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-70215d2f-df67-4cb0-8495-e7ad54257309 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:09:01.692454 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9986d8-7dce-4880-b685-4f1aa55b8945 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:01.692830 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9986d8-7dce-4880-b685-4f1aa55b8945 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:01.693048 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9986d8-7dce-4880-b685-4f1aa55b8945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:01.718343 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-2e9986d8-7dce-4880-b685-4f1aa55b8945 None None] MFA Rules not processed for user `ff5cfe4d31614e2db9059c69116297cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:09:01.720107 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2e9986d8-7dce-4880-b685-4f1aa55b8945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:09:01.737954 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70215d2f-df67-4cb0-8495-e7ad54257309 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:09:01.744345 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1dc15a-919d-403e-a167-6eb838887f0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:09:01.745184 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1dc15a-919d-403e-a167-6eb838887f0c service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:01.745566 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1dc15a-919d-403e-a167-6eb838887f0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:01.745943 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1dc15a-919d-403e-a167-6eb838887f0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:01.758108 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1dc15a-919d-403e-a167-6eb838887f0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:09:01.759735 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1dc15a-919d-403e-a167-6eb838887f0c service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:09:01.770787 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1017/2018] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:09:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:09:01.792183 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1002/2019] 10.222.0.13 () {58 vars in 967 bytes} [Sun Oct 23 20:09:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:09:01.805715 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1dc15a-919d-403e-a167-6eb838887f0c service neutron] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:09:01.807365 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-3d1dc15a-919d-403e-a167-6eb838887f0c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:09:01.880715 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1dc15a-919d-403e-a167-6eb838887f0c service neutron] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:09:01.886143 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f45de2-1236-4c78-af1b-1939b8e8756b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15f603d1e7f54751aa79317e2ea00a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:09:01.886920 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f45de2-1236-4c78-af1b-1939b8e8756b service swift] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:01.887306 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f45de2-1236-4c78-af1b-1939b8e8756b service swift] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:01.887646 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f45de2-1236-4c78-af1b-1939b8e8756b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:01.896415 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f45de2-1236-4c78-af1b-1939b8e8756b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:09:01.898070 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f45de2-1236-4c78-af1b-1939b8e8756b service swift] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:09:01.907328 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1018/2020] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Oct 23 20:09:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 20:09:03.983007 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9986d8-7dce-4880-b685-4f1aa55b8945 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:09:03.984492 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2e9986d8-7dce-4880-b685-4f1aa55b8945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:09:04.004245 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9986d8-7dce-4880-b685-4f1aa55b8945 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:09:04.012383 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698f3ca1-87e3-422f-ab36-2c0a43f1a013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:09:04.013431 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f3ca1-87e3-422f-ab36-2c0a43f1a013 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:04.014020 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f3ca1-87e3-422f-ab36-2c0a43f1a013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:04.015109 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f3ca1-87e3-422f-ab36-2c0a43f1a013 admin admin] PATH_INFO: `/v3/users/48be37e3311a4d6c968b135f8e5ee536` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:04.018622 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698f3ca1-87e3-422f-ab36-2c0a43f1a013 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48be37e3311a4d6c968b135f8e5ee536)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:09:04.024254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698f3ca1-87e3-422f-ab36-2c0a43f1a013 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:09:04.089949 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-698f3ca1-87e3-422f-ab36-2c0a43f1a013 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48be37e3311a4d6c968b135f8e5ee536'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:04.105328 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-698f3ca1-87e3-422f-ab36-2c0a43f1a013 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48be37e3311a4d6c968b135f8e5ee536'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:04.115183 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-698f3ca1-87e3-422f-ab36-2c0a43f1a013 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48be37e3311a4d6c968b135f8e5ee536'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:04.116038 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-698f3ca1-87e3-422f-ab36-2c0a43f1a013 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48be37e3311a4d6c968b135f8e5ee536'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:04.124872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-698f3ca1-87e3-422f-ab36-2c0a43f1a013 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48be37e3311a4d6c968b135f8e5ee536'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:04.134748 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1003/2021] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:09:03 2022] DELETE /identity/v3/users/48be37e3311a4d6c968b135f8e5ee536 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:09:04.350729 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f45de2-1236-4c78-af1b-1939b8e8756b service swift] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:09:04.370085 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f45de2-1236-4c78-af1b-1939b8e8756b service swift] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:09:04.417290 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:09:04.442671 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d65e8c-4333-42ad-a844-be1d5fac5056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:09:04.443322 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d65e8c-4333-42ad-a844-be1d5fac5056 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:04.443603 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d65e8c-4333-42ad-a844-be1d5fac5056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:04.444404 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d65e8c-4333-42ad-a844-be1d5fac5056 admin admin] PATH_INFO: `/v3/projects/46b77aad0457485693b9a43b85506915` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:04.446055 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d65e8c-4333-42ad-a844-be1d5fac5056 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46b77aad0457485693b9a43b85506915)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:09:04.449496 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d65e8c-4333-42ad-a844-be1d5fac5056 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:09:04.507653 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-77d65e8c-4333-42ad-a844-be1d5fac5056 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46b77aad0457485693b9a43b85506915', 'request_id': 'req-77d65e8c-4333-42ad-a844-be1d5fac5056'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:04.516828 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-77d65e8c-4333-42ad-a844-be1d5fac5056 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46b77aad0457485693b9a43b85506915', 'request_id': 'req-77d65e8c-4333-42ad-a844-be1d5fac5056'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:04.524686 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-77d65e8c-4333-42ad-a844-be1d5fac5056 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46b77aad0457485693b9a43b85506915', 'request_id': 'req-77d65e8c-4333-42ad-a844-be1d5fac5056'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:04.537452 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-77d65e8c-4333-42ad-a844-be1d5fac5056 admin admin] The token cache is being invalidate because project 46b77aad0457485693b9a43b85506915 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:09:04.537452 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-77d65e8c-4333-42ad-a844-be1d5fac5056 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:04.538120 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-77d65e8c-4333-42ad-a844-be1d5fac5056 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:04.542319 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1019/2022] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:09:04 2022] DELETE /identity/v3/projects/46b77aad0457485693b9a43b85506915 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:09:04.544661 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2d163-3bb5-4c89-8103-8d31ba35b633 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:04.544956 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2d163-3bb5-4c89-8103-8d31ba35b633 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:04.545235 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2d163-3bb5-4c89-8103-8d31ba35b633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:04.566102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-cfc2d163-3bb5-4c89-8103-8d31ba35b633 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:09:04.566479 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-cfc2d163-3bb5-4c89-8103-8d31ba35b633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:09:04.612854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:09:04.654286 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1004/2023] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:09:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:09:04.655260 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd11dc-e353-40b4-82ce-f502bdc467e7 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:04.655550 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd11dc-e353-40b4-82ce-f502bdc467e7 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:04.655824 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7dd11dc-e353-40b4-82ce-f502bdc467e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:04.676852 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-a7dd11dc-e353-40b4-82ce-f502bdc467e7 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:09:04.677980 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-a7dd11dc-e353-40b4-82ce-f502bdc467e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:09:04.687316 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1020/2024] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:09:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:09:04.691347 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc2d163-3bb5-4c89-8103-8d31ba35b633 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:09:04.705358 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc2d163-3bb5-4c89-8103-8d31ba35b633 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:09:04.712611 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b27b60-0400-4cf8-a5d9-5f58ccf40fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:09:04.713241 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b27b60-0400-4cf8-a5d9-5f58ccf40fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:04.713525 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b27b60-0400-4cf8-a5d9-5f58ccf40fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:04.713779 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b27b60-0400-4cf8-a5d9-5f58ccf40fb1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:04.714474 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b27b60-0400-4cf8-a5d9-5f58ccf40fb1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:09:04.714777 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b27b60-0400-4cf8-a5d9-5f58ccf40fb1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:09:04.716387 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b27b60-0400-4cf8-a5d9-5f58ccf40fb1 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:09:04.731695 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1005/2025] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:09:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:09:04.735645 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd11dc-e353-40b4-82ce-f502bdc467e7 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:09:04.749087 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7dd11dc-e353-40b4-82ce-f502bdc467e7 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:09:04.755848 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d531c63b-e414-492a-b0c9-b6edd940f69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:09:04.757011 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531c63b-e414-492a-b0c9-b6edd940f69c admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:04.757479 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531c63b-e414-492a-b0c9-b6edd940f69c admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:04.757902 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d531c63b-e414-492a-b0c9-b6edd940f69c admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:04.758774 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d531c63b-e414-492a-b0c9-b6edd940f69c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:09:04.761242 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d531c63b-e414-492a-b0c9-b6edd940f69c admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:09:04.800428 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1021/2026] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:09:04 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:09:04.804788 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b27b60-0400-4cf8-a5d9-5f58ccf40fb1 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:09:04.820021 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b27b60-0400-4cf8-a5d9-5f58ccf40fb1 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:09:04.906490 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b2aee4-5f63-405e-824b-99d580ac3432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:09:04.907165 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b2aee4-5f63-405e-824b-99d580ac3432 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:04.907471 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b2aee4-5f63-405e-824b-99d580ac3432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:04.907703 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b2aee4-5f63-405e-824b-99d580ac3432 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:04.908359 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b2aee4-5f63-405e-824b-99d580ac3432 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:09:04.912175 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b2aee4-5f63-405e-824b-99d580ac3432 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:09:04.939303 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1006/2027] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:09:04 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:09:04.944144 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d531c63b-e414-492a-b0c9-b6edd940f69c admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:09:04.955021 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d531c63b-e414-492a-b0c9-b6edd940f69c admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:09:04.960436 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f9a94-f5f9-4ff8-b342-dffa779eee62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:09:04.961054 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f9a94-f5f9-4ff8-b342-dffa779eee62 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:04.961337 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f9a94-f5f9-4ff8-b342-dffa779eee62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:04.961615 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339f9a94-f5f9-4ff8-b342-dffa779eee62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:04.962297 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f9a94-f5f9-4ff8-b342-dffa779eee62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:09:04.963952 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339f9a94-f5f9-4ff8-b342-dffa779eee62 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:09:04.975741 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1022/2028] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:09:04.979694 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b2aee4-5f63-405e-824b-99d580ac3432 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:09:04.991091 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b2aee4-5f63-405e-824b-99d580ac3432 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:09:04.996272 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e550ad9-d471-4e43-bc57-8fe75e9d2c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:09:04.997876 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e550ad9-d471-4e43-bc57-8fe75e9d2c4e admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:04.998225 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e550ad9-d471-4e43-bc57-8fe75e9d2c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:04.998923 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e550ad9-d471-4e43-bc57-8fe75e9d2c4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:04.999435 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e550ad9-d471-4e43-bc57-8fe75e9d2c4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:09:05.001637 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e550ad9-d471-4e43-bc57-8fe75e9d2c4e admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:09:05.018384 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1007/2029] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:09:05.035216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f9a94-f5f9-4ff8-b342-dffa779eee62 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:09:05.051022 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339f9a94-f5f9-4ff8-b342-dffa779eee62 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:09:05.061186 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaab03e-efc7-4fa4-bc6e-d5493ca14553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:09:05.062093 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaab03e-efc7-4fa4-bc6e-d5493ca14553 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:05.063095 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaab03e-efc7-4fa4-bc6e-d5493ca14553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:05.063626 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaab03e-efc7-4fa4-bc6e-d5493ca14553 admin admin] PATH_INFO: `/v3/projects/1265b40c8a364e49a3b7a9b15c04325f/users/27597fde728d4bb8ba2852f829e7c07e/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:05.097053 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaab03e-efc7-4fa4-bc6e-d5493ca14553 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1265b40c8a364e49a3b7a9b15c04325f, user_id=27597fde728d4bb8ba2852f829e7c07e, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:09:05.103672 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaab03e-efc7-4fa4-bc6e-d5493ca14553 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:09:05.151433 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1023/2030] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:09:05 2022] PUT /identity/v3/projects/1265b40c8a364e49a3b7a9b15c04325f/users/27597fde728d4bb8ba2852f829e7c07e/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:09:05.163255 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e6ccd-bb03-497f-8962-d3dfdbecfb4e None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:05.163948 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e6ccd-bb03-497f-8962-d3dfdbecfb4e None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:05.164473 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e6ccd-bb03-497f-8962-d3dfdbecfb4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:05.221172 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-a85e6ccd-bb03-497f-8962-d3dfdbecfb4e None None] MFA Rules not processed for user `27597fde728d4bb8ba2852f829e7c07e`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:09:05.222806 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a85e6ccd-bb03-497f-8962-d3dfdbecfb4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:09:05.318794 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1008/2031] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:09:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:09:05.333187 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaab03e-efc7-4fa4-bc6e-d5493ca14553 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:09:05.335362 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-ceaab03e-efc7-4fa4-bc6e-d5493ca14553 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:09:05.408656 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaab03e-efc7-4fa4-bc6e-d5493ca14553 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:09:05.414708 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87243a3-9676-4840-bf4b-977912779747 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:09:05.415344 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87243a3-9676-4840-bf4b-977912779747 service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:05.415920 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87243a3-9676-4840-bf4b-977912779747 service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:05.416210 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87243a3-9676-4840-bf4b-977912779747 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:05.427926 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87243a3-9676-4840-bf4b-977912779747 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:09:05.430135 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87243a3-9676-4840-bf4b-977912779747 service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:09:05.441326 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1024/2032] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:09:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:09:05.945920 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19524400-5c43-4284-ba5c-ae2d08816ca5 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:05.947831 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19524400-5c43-4284-ba5c-ae2d08816ca5 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:05.948146 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19524400-5c43-4284-ba5c-ae2d08816ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:05.982366 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-19524400-5c43-4284-ba5c-ae2d08816ca5 None None] MFA Rules not processed for user `21fac7ecf4f2435483df150292a0052b`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:09:05.983907 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-19524400-5c43-4284-ba5c-ae2d08816ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:09:06.005378 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:09:06.054876 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1009/2033] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:09:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:09:06.059541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87243a3-9676-4840-bf4b-977912779747 service cinder] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:09:06.060948 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-b87243a3-9676-4840-bf4b-977912779747 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:09:06.074739 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:09:06.136157 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87243a3-9676-4840-bf4b-977912779747 service cinder] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:09:06.142218 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e75e15-f34b-4785-b95d-e469942be3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:09:06.142891 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e75e15-f34b-4785-b95d-e469942be3b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:06.143174 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e75e15-f34b-4785-b95d-e469942be3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:06.143449 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e75e15-f34b-4785-b95d-e469942be3b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:06.144094 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e75e15-f34b-4785-b95d-e469942be3b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:09:06.146058 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e75e15-f34b-4785-b95d-e469942be3b5 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:09:06.178358 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1025/2034] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:09:06 2022] POST /identity/v3/projects => generated 369 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:09:06.182414 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19524400-5c43-4284-ba5c-ae2d08816ca5 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:09:06.197332 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19524400-5c43-4284-ba5c-ae2d08816ca5 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:09:06.262893 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a00f7-1197-4a94-82c8-49721cfa487f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:09:06.263937 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a00f7-1197-4a94-82c8-49721cfa487f admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:06.264232 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a00f7-1197-4a94-82c8-49721cfa487f admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:06.264519 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a00f7-1197-4a94-82c8-49721cfa487f admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:06.265189 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a00f7-1197-4a94-82c8-49721cfa487f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:09:06.267280 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a00f7-1197-4a94-82c8-49721cfa487f admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:09:06.293426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1010/2035] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:09:06 2022] POST /identity/v3/users => generated 291 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:09:06.296803 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e75e15-f34b-4785-b95d-e469942be3b5 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:09:06.307603 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e75e15-f34b-4785-b95d-e469942be3b5 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:09:06.316065 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d6b1ff-1511-403a-a798-23652816d3de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:09:06.316695 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6b1ff-1511-403a-a798-23652816d3de admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:06.316978 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6b1ff-1511-403a-a798-23652816d3de admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:06.317429 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6b1ff-1511-403a-a798-23652816d3de admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:06.318713 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d6b1ff-1511-403a-a798-23652816d3de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:09:06.321111 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d6b1ff-1511-403a-a798-23652816d3de admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:09:06.334436 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1026/2036] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:09:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:09:06.338670 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a00f7-1197-4a94-82c8-49721cfa487f admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:09:06.349530 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a00f7-1197-4a94-82c8-49721cfa487f admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:09:06.354278 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d56a09-3491-4bc8-99df-d486dab5e03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:09:06.354972 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d56a09-3491-4bc8-99df-d486dab5e03c admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:06.355251 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d56a09-3491-4bc8-99df-d486dab5e03c admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:06.355526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d56a09-3491-4bc8-99df-d486dab5e03c admin admin] PATH_INFO: `/v3/projects/b81466a34db6497a94e114345a37e183/users/916e488424c447e4ac51df6c46989499/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:06.371609 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d56a09-3491-4bc8-99df-d486dab5e03c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b81466a34db6497a94e114345a37e183, user_id=916e488424c447e4ac51df6c46989499, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:09:06.374402 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d56a09-3491-4bc8-99df-d486dab5e03c admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:09:06.388578 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1011/2037] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:09:06 2022] PUT /identity/v3/projects/b81466a34db6497a94e114345a37e183/users/916e488424c447e4ac51df6c46989499/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:09:06.392487 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d6b1ff-1511-403a-a798-23652816d3de admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:09:06.403587 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d6b1ff-1511-403a-a798-23652816d3de admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:09:06.465312 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6da0d87-9fb1-4a52-89d0-bd00587728bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:09:06.465965 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da0d87-9fb1-4a52-89d0-bd00587728bf admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:06.466320 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da0d87-9fb1-4a52-89d0-bd00587728bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:06.466616 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6da0d87-9fb1-4a52-89d0-bd00587728bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:06.467242 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6da0d87-9fb1-4a52-89d0-bd00587728bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:09:06.468835 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6da0d87-9fb1-4a52-89d0-bd00587728bf admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:09:06.480281 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1027/2038] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:09:06 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:09:06.484318 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d56a09-3491-4bc8-99df-d486dab5e03c admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:09:06.495503 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d56a09-3491-4bc8-99df-d486dab5e03c admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:09:06.500615 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92df062-c157-4678-9216-3d249bb884f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:09:06.501319 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92df062-c157-4678-9216-3d249bb884f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:06.501600 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92df062-c157-4678-9216-3d249bb884f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:06.501873 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92df062-c157-4678-9216-3d249bb884f8 admin admin] PATH_INFO: `/v3/projects/b81466a34db6497a94e114345a37e183/users/916e488424c447e4ac51df6c46989499/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:06.503920 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92df062-c157-4678-9216-3d249bb884f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b81466a34db6497a94e114345a37e183, user_id=916e488424c447e4ac51df6c46989499, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:09:06.506749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92df062-c157-4678-9216-3d249bb884f8 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:09:06.518016 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1012/2039] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:09:06 2022] PUT /identity/v3/projects/b81466a34db6497a94e114345a37e183/users/916e488424c447e4ac51df6c46989499/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:09:08.194726 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6da0d87-9fb1-4a52-89d0-bd00587728bf admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:09:08.228709 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6da0d87-9fb1-4a52-89d0-bd00587728bf admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:09:08.331428 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27af0258-f88e-4721-91bb-f90ea65c0dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:09:08.332373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af0258-f88e-4721-91bb-f90ea65c0dad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:08.332857 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af0258-f88e-4721-91bb-f90ea65c0dad admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:08.334164 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27af0258-f88e-4721-91bb-f90ea65c0dad admin admin] PATH_INFO: `/v3/users/27597fde728d4bb8ba2852f829e7c07e` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:08.336249 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27af0258-f88e-4721-91bb-f90ea65c0dad admin admin] RBAC: Authorizing `identity:delete_user(user_id=27597fde728d4bb8ba2852f829e7c07e)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:09:08.339136 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27af0258-f88e-4721-91bb-f90ea65c0dad admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:09:08.378222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:09:08.408279 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-27af0258-f88e-4721-91bb-f90ea65c0dad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27597fde728d4bb8ba2852f829e7c07e'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:08.424271 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-27af0258-f88e-4721-91bb-f90ea65c0dad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27597fde728d4bb8ba2852f829e7c07e'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:08.433967 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-27af0258-f88e-4721-91bb-f90ea65c0dad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27597fde728d4bb8ba2852f829e7c07e'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:08.442499 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-27af0258-f88e-4721-91bb-f90ea65c0dad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27597fde728d4bb8ba2852f829e7c07e'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:08.444130 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-27af0258-f88e-4721-91bb-f90ea65c0dad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27597fde728d4bb8ba2852f829e7c07e'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:08.453533 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1028/2040] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:09:08 2022] DELETE /identity/v3/users/27597fde728d4bb8ba2852f829e7c07e => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:09:08.458757 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a97c8-52e2-481f-aa73-ff4ab9b2b546 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:08.459266 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a97c8-52e2-481f-aa73-ff4ab9b2b546 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:08.459359 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8a97c8-52e2-481f-aa73-ff4ab9b2b546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:08.479864 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-8f8a97c8-52e2-481f-aa73-ff4ab9b2b546 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:09:08.481089 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-8f8a97c8-52e2-481f-aa73-ff4ab9b2b546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:09:08.559701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1013/2041] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:09:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:09:08.561282 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27af0258-f88e-4721-91bb-f90ea65c0dad admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:09:08.562832 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-27af0258-f88e-4721-91bb-f90ea65c0dad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:09:08.632209 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27af0258-f88e-4721-91bb-f90ea65c0dad admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:09:08.636128 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496af8a5-db72-4dfc-8975-97c87268047f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:09:08.636749 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496af8a5-db72-4dfc-8975-97c87268047f service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:08.637033 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496af8a5-db72-4dfc-8975-97c87268047f service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:08.637307 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496af8a5-db72-4dfc-8975-97c87268047f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:08.647710 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496af8a5-db72-4dfc-8975-97c87268047f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:09:08.649150 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496af8a5-db72-4dfc-8975-97c87268047f service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:09:08.659359 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1029/2042] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:09:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:09:09.087589 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a97c8-52e2-481f-aa73-ff4ab9b2b546 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:09:09.088979 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-8f8a97c8-52e2-481f-aa73-ff4ab9b2b546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:09:09.112005 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8a97c8-52e2-481f-aa73-ff4ab9b2b546 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:09:09.124731 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02320d55-f35e-491a-adf4-30fa1d4abcb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:09:09.125399 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02320d55-f35e-491a-adf4-30fa1d4abcb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:09.125704 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02320d55-f35e-491a-adf4-30fa1d4abcb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:09.125962 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02320d55-f35e-491a-adf4-30fa1d4abcb0 admin admin] PATH_INFO: `/v3/projects/1265b40c8a364e49a3b7a9b15c04325f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:09.127182 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02320d55-f35e-491a-adf4-30fa1d4abcb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1265b40c8a364e49a3b7a9b15c04325f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:09:09.129083 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02320d55-f35e-491a-adf4-30fa1d4abcb0 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:09:09.142795 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:09:09.216052 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-02320d55-f35e-491a-adf4-30fa1d4abcb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1265b40c8a364e49a3b7a9b15c04325f', 'request_id': 'req-02320d55-f35e-491a-adf4-30fa1d4abcb0'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:09.227569 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-02320d55-f35e-491a-adf4-30fa1d4abcb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1265b40c8a364e49a3b7a9b15c04325f', 'request_id': 'req-02320d55-f35e-491a-adf4-30fa1d4abcb0'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:09.235467 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-02320d55-f35e-491a-adf4-30fa1d4abcb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1265b40c8a364e49a3b7a9b15c04325f', 'request_id': 'req-02320d55-f35e-491a-adf4-30fa1d4abcb0'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:09.257817 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-02320d55-f35e-491a-adf4-30fa1d4abcb0 admin admin] The token cache is being invalidate because project 1265b40c8a364e49a3b7a9b15c04325f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:09:09.258449 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-02320d55-f35e-491a-adf4-30fa1d4abcb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:09.259964 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-02320d55-f35e-491a-adf4-30fa1d4abcb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:09:09.263088 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1014/2043] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:09:09 2022] DELETE /identity/v3/projects/1265b40c8a364e49a3b7a9b15c04325f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:09:10.929841 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0833d8-ea2b-4ab1-89c8-fef821671ccf None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:10.930365 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0833d8-ea2b-4ab1-89c8-fef821671ccf None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:10.930665 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0833d8-ea2b-4ab1-89c8-fef821671ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:10.958761 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-0d0833d8-ea2b-4ab1-89c8-fef821671ccf None None] MFA Rules not processed for user `916e488424c447e4ac51df6c46989499`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:09:10.959996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-0d0833d8-ea2b-4ab1-89c8-fef821671ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:09:11.036484 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1030/2044] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:09:10 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:09:11.053525 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02320d55-f35e-491a-adf4-30fa1d4abcb0 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:09:11.055135 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-02320d55-f35e-491a-adf4-30fa1d4abcb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:09:11.153465 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02320d55-f35e-491a-adf4-30fa1d4abcb0 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:09:11.161953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f15c04e-24b0-447a-8049-81688b2336d4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:09:11.162710 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15c04e-24b0-447a-8049-81688b2336d4 service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:11.163011 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15c04e-24b0-447a-8049-81688b2336d4 service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:11.163512 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f15c04e-24b0-447a-8049-81688b2336d4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:11.165358 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5f15c04e-24b0-447a-8049-81688b2336d4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:09:11.226914 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f15c04e-24b0-447a-8049-81688b2336d4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:09:11.228536 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f15c04e-24b0-447a-8049-81688b2336d4 service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:09:11.237907 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1015/2045] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:09:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:09:12.321860 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd2386-6f8d-4ccf-943a-e0922e92dd2d None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:12.322359 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd2386-6f8d-4ccf-943a-e0922e92dd2d None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:12.322455 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd2386-6f8d-4ccf-943a-e0922e92dd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:12.341758 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-e2fd2386-6f8d-4ccf-943a-e0922e92dd2d None None] MFA Rules not processed for user `21fac7ecf4f2435483df150292a0052b`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:09:12.343014 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-e2fd2386-6f8d-4ccf-943a-e0922e92dd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:09:12.349874 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1031/2046] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:09:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:09:21.233568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=77,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:09:22.336054 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=83,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:09:44.671004 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be11721-5ce2-4b1a-9ee5-e2764b46fd71 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:44.671846 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be11721-5ce2-4b1a-9ee5-e2764b46fd71 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:44.672156 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be11721-5ce2-4b1a-9ee5-e2764b46fd71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:44.693054 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-6be11721-5ce2-4b1a-9ee5-e2764b46fd71 None None] MFA Rules not processed for user `ff5cfe4d31614e2db9059c69116297cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:09:44.694668 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-6be11721-5ce2-4b1a-9ee5-e2764b46fd71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:09:44.752519 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1016/2047] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Oct 23 20:09:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:09:44.775499 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd2386-6f8d-4ccf-943a-e0922e92dd2d None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:09:44.776933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-e2fd2386-6f8d-4ccf-943a-e0922e92dd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:09:44.846869 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd2386-6f8d-4ccf-943a-e0922e92dd2d None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:09:44.853162 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e589c2e-175e-45f7-a3a4-2f771450d496 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15f603d1e7f54751aa79317e2ea00a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:09:44.854444 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e589c2e-175e-45f7-a3a4-2f771450d496 service swift] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:44.854928 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e589c2e-175e-45f7-a3a4-2f771450d496 service swift] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:44.855212 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e589c2e-175e-45f7-a3a4-2f771450d496 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:44.863942 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e589c2e-175e-45f7-a3a4-2f771450d496 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:09:44.865533 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e589c2e-175e-45f7-a3a4-2f771450d496 service swift] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:09:44.873796 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1032/2048] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Oct 23 20:09:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 20:09:54.745619 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=25 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:09:54.870385 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=32 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:09:56.062125 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be11721-5ce2-4b1a-9ee5-e2764b46fd71 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:09:56.063605 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-6be11721-5ce2-4b1a-9ee5-e2764b46fd71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:09:56.118668 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be11721-5ce2-4b1a-9ee5-e2764b46fd71 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:09:56.122159 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2265dc3d-69b7-423b-b2b8-7714a84713c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:09:56.122793 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265dc3d-69b7-423b-b2b8-7714a84713c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:09:56.123069 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265dc3d-69b7-423b-b2b8-7714a84713c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:09:56.123343 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265dc3d-69b7-423b-b2b8-7714a84713c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:09:56.132048 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2265dc3d-69b7-423b-b2b8-7714a84713c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:09:56.133982 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2265dc3d-69b7-423b-b2b8-7714a84713c1 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:09:56.143533 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1017/2049] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:09:56 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:10:00.050972 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ed61f-9b8d-4a16-8ba9-1701a5af78d9 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:10:00.051588 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ed61f-9b8d-4a16-8ba9-1701a5af78d9 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:10:00.051694 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7ed61f-9b8d-4a16-8ba9-1701a5af78d9 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:10:00.053154 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1033/2050] 10.222.0.13 () {54 vars in 921 bytes} [Sun Oct 23 20:10:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:10:00.060429 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa4b4b-fef2-4712-84c5-5f0424b40175 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:10:00.060891 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa4b4b-fef2-4712-84c5-5f0424b40175 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:10:00.061013 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa4b4b-fef2-4712-84c5-5f0424b40175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:10:00.080737 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-2dfa4b4b-fef2-4712-84c5-5f0424b40175 None None] MFA Rules not processed for user `0b3d512304254ffcaa8dad90abf1f9c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:10:00.081939 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2dfa4b4b-fef2-4712-84c5-5f0424b40175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:10:00.092315 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1018/2051] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Oct 23 20:10:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:10:00.104136 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ed61f-9b8d-4a16-8ba9-1701a5af78d9 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:10:00.105812 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-ce7ed61f-9b8d-4a16-8ba9-1701a5af78d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:10:00.163145 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7ed61f-9b8d-4a16-8ba9-1701a5af78d9 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:10:00.167418 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aee8b8-8444-403b-a2af-9e2ba1f407d8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '021216f90d7f4ff6b6eb7a1d86399d99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:10:00.168090 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aee8b8-8444-403b-a2af-9e2ba1f407d8 service glance] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:10:00.168433 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aee8b8-8444-403b-a2af-9e2ba1f407d8 service glance] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:10:00.168767 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aee8b8-8444-403b-a2af-9e2ba1f407d8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:10:00.182223 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aee8b8-8444-403b-a2af-9e2ba1f407d8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:10:00.183689 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aee8b8-8444-403b-a2af-9e2ba1f407d8 service glance] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:10:00.194526 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1034/2052] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:10:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:10:00.263259 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121586a4-56f0-45be-a75a-b1e13427c946 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:10:00.263547 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121586a4-56f0-45be-a75a-b1e13427c946 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:10:00.263855 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121586a4-56f0-45be-a75a-b1e13427c946 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:10:00.265202 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1019/2053] 10.222.0.13 () {54 vars in 921 bytes} [Sun Oct 23 20:10:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:10:00.270121 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f5779-bad9-4111-bdf5-4bb06c6bf14d None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:10:00.270437 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f5779-bad9-4111-bdf5-4bb06c6bf14d None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:10:00.270709 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3f5779-bad9-4111-bdf5-4bb06c6bf14d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:10:00.293080 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-ec3f5779-bad9-4111-bdf5-4bb06c6bf14d None None] MFA Rules not processed for user `ba8ff3413bc549d29486d1c724a1f383`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:10:00.294296 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-ec3f5779-bad9-4111-bdf5-4bb06c6bf14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:10:00.300875 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1035/2054] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Oct 23 20:10:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:10:00.330467 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121586a4-56f0-45be-a75a-b1e13427c946 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:10:00.341856 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121586a4-56f0-45be-a75a-b1e13427c946 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:10:00.345538 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37cb49e-4926-4f37-b0c3-18a9409bf78c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:10:00.346207 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37cb49e-4926-4f37-b0c3-18a9409bf78c service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:10:00.346579 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37cb49e-4926-4f37-b0c3-18a9409bf78c service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:10:00.346919 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37cb49e-4926-4f37-b0c3-18a9409bf78c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:10:00.354902 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37cb49e-4926-4f37-b0c3-18a9409bf78c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:10:00.356424 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37cb49e-4926-4f37-b0c3-18a9409bf78c service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:10:00.364884 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1020/2055] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:10:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:10:00.474052 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f869a73-deb5-4220-84db-7678351d509f None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:10:00.474369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f869a73-deb5-4220-84db-7678351d509f None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:10:00.474640 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f869a73-deb5-4220-84db-7678351d509f None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:10:00.475997 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1036/2056] 10.222.0.13 () {54 vars in 921 bytes} [Sun Oct 23 20:10:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:10:00.480810 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff185505-888b-4fd2-8114-4c4da83e7c60 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:10:00.481097 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff185505-888b-4fd2-8114-4c4da83e7c60 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:10:00.481376 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff185505-888b-4fd2-8114-4c4da83e7c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:10:00.500706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-ff185505-888b-4fd2-8114-4c4da83e7c60 None None] MFA Rules not processed for user `0b3d512304254ffcaa8dad90abf1f9c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:10:00.501901 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ff185505-888b-4fd2-8114-4c4da83e7c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:10:00.511571 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1021/2057] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Oct 23 20:10:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:10:00.521591 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f869a73-deb5-4220-84db-7678351d509f None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:10:00.532927 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f869a73-deb5-4220-84db-7678351d509f None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:10:00.536476 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff41070-a478-4f14-849d-bc31699ade5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:10:00.537080 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff41070-a478-4f14-849d-bc31699ade5f service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:10:00.537340 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff41070-a478-4f14-849d-bc31699ade5f service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:10:00.537605 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff41070-a478-4f14-849d-bc31699ade5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:10:00.547847 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff41070-a478-4f14-849d-bc31699ade5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:10:00.549267 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff41070-a478-4f14-849d-bc31699ade5f service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:10:00.559888 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1037/2058] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:10:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 20:10:03.099300 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03b2c8-d7b8-4da9-91a7-789c296a7728 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:10:03.099776 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03b2c8-d7b8-4da9-91a7-789c296a7728 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:10:03.099881 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e03b2c8-d7b8-4da9-91a7-789c296a7728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:10:03.119361 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-6e03b2c8-d7b8-4da9-91a7-789c296a7728 None None] MFA Rules not processed for user `ff5cfe4d31614e2db9059c69116297cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:10:03.120554 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-6e03b2c8-d7b8-4da9-91a7-789c296a7728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:10:03.127640 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1022/2059] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Oct 23 20:10:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:10:03.141216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff41070-a478-4f14-849d-bc31699ade5f service neutron] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:10:03.152575 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff41070-a478-4f14-849d-bc31699ade5f service neutron] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:10:03.157518 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19814d96-4f6e-4c46-b710-6816dfebeb18 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15f603d1e7f54751aa79317e2ea00a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:10:03.158140 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19814d96-4f6e-4c46-b710-6816dfebeb18 service swift] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:10:03.158439 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19814d96-4f6e-4c46-b710-6816dfebeb18 service swift] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:10:03.158710 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19814d96-4f6e-4c46-b710-6816dfebeb18 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:10:03.166808 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19814d96-4f6e-4c46-b710-6816dfebeb18 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:10:03.168459 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19814d96-4f6e-4c46-b710-6816dfebeb18 service swift] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:10:03.176499 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1038/2060] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Oct 23 20:10:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 23 20:10:11.280451 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03b2c8-d7b8-4da9-91a7-789c296a7728 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:10:11.304416 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e03b2c8-d7b8-4da9-91a7-789c296a7728 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:10:11.308771 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04cf3f-f3e6-408d-a4f1-5f1137e142a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:10:11.309379 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04cf3f-f3e6-408d-a4f1-5f1137e142a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:10:11.309653 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04cf3f-f3e6-408d-a4f1-5f1137e142a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:10:11.309922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04cf3f-f3e6-408d-a4f1-5f1137e142a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:10:11.311663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4e04cf3f-f3e6-408d-a4f1-5f1137e142a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:10:11.325509 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e04cf3f-f3e6-408d-a4f1-5f1137e142a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:10:11.326958 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e04cf3f-f3e6-408d-a4f1-5f1137e142a7 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:10:11.339409 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1023/2061] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:10:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:10:13.173371 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=41 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:10:13.640163 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19814d96-4f6e-4c46-b710-6816dfebeb18 service swift] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:10:13.651066 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19814d96-4f6e-4c46-b710-6816dfebeb18 service swift] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:10:13.654766 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f030ef3-aaef-4645-893e-8c2b35ee21e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:10:13.655394 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f030ef3-aaef-4645-893e-8c2b35ee21e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:10:13.655673 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f030ef3-aaef-4645-893e-8c2b35ee21e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:10:13.655944 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f030ef3-aaef-4645-893e-8c2b35ee21e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:10:13.657673 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-0f030ef3-aaef-4645-893e-8c2b35ee21e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:10:13.672781 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f030ef3-aaef-4645-893e-8c2b35ee21e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:10:13.674220 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f030ef3-aaef-4645-893e-8c2b35ee21e0 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:10:13.685900 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1039/2062] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:10:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:10:21.333094 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=47 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:10:23.681436 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=6 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:10:25.791203 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04cf3f-f3e6-408d-a4f1-5f1137e142a7 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:10:25.792674 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4e04cf3f-f3e6-408d-a4f1-5f1137e142a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:10:25.806165 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04cf3f-f3e6-408d-a4f1-5f1137e142a7 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:10:25.811636 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb02a13f-441e-4b46-98c4-84c38999a7e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:10:25.812235 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02a13f-441e-4b46-98c4-84c38999a7e3 service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:10:25.812516 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02a13f-441e-4b46-98c4-84c38999a7e3 service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:10:25.812792 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02a13f-441e-4b46-98c4-84c38999a7e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:10:25.814461 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-fb02a13f-441e-4b46-98c4-84c38999a7e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:10:25.872509 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb02a13f-441e-4b46-98c4-84c38999a7e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:10:25.874080 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb02a13f-441e-4b46-98c4-84c38999a7e3 service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:10:25.883076 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1024/2063] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:10:25 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:10:33.688294 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97520a48-8e17-46b0-972c-f6b69e543c55 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:10:33.688786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97520a48-8e17-46b0-972c-f6b69e543c55 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:10:33.688879 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97520a48-8e17-46b0-972c-f6b69e543c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:10:33.707653 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-97520a48-8e17-46b0-972c-f6b69e543c55 None None] MFA Rules not processed for user `ff5cfe4d31614e2db9059c69116297cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:10:33.708807 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-97520a48-8e17-46b0-972c-f6b69e543c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:10:33.715770 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1040/2064] 10.222.0.13 () {58 vars in 967 bytes} [Sun Oct 23 20:10:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:10:33.728251 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb02a13f-441e-4b46-98c4-84c38999a7e3 service nova] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:10:33.740051 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb02a13f-441e-4b46-98c4-84c38999a7e3 service nova] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:10:33.746368 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78dc85-a2a3-42a8-9b5a-8b0ecae2d587 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15f603d1e7f54751aa79317e2ea00a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:10:33.746985 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78dc85-a2a3-42a8-9b5a-8b0ecae2d587 service swift] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:10:33.747268 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78dc85-a2a3-42a8-9b5a-8b0ecae2d587 service swift] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:10:33.747541 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d78dc85-a2a3-42a8-9b5a-8b0ecae2d587 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:10:33.755958 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d78dc85-a2a3-42a8-9b5a-8b0ecae2d587 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:10:33.757530 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d78dc85-a2a3-42a8-9b5a-8b0ecae2d587 service swift] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:10:33.765532 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1025/2065] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Oct 23 20:10:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 20:10:41.924469 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97520a48-8e17-46b0-972c-f6b69e543c55 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:10:41.937400 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97520a48-8e17-46b0-972c-f6b69e543c55 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:10:41.943116 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293126bb-ba32-4ce7-926e-aab964342f46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:10:41.943782 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293126bb-ba32-4ce7-926e-aab964342f46 service nova] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:10:41.944175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293126bb-ba32-4ce7-926e-aab964342f46 service nova] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:10:41.944454 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293126bb-ba32-4ce7-926e-aab964342f46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:10:41.947048 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-293126bb-ba32-4ce7-926e-aab964342f46 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:10:42.003803 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293126bb-ba32-4ce7-926e-aab964342f46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:10:42.005384 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293126bb-ba32-4ce7-926e-aab964342f46 service nova] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:10:42.014986 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1041/2066] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:10:41 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:10:43.762388 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=34 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:10:50.066187 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78dc85-a2a3-42a8-9b5a-8b0ecae2d587 service swift] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:10:50.067703 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-2d78dc85-a2a3-42a8-9b5a-8b0ecae2d587 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:10:50.080096 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d78dc85-a2a3-42a8-9b5a-8b0ecae2d587 service swift] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:10:50.083773 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a66b86-d9f5-452b-b3f2-a8f3bb2ff032 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:10:50.084389 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a66b86-d9f5-452b-b3f2-a8f3bb2ff032 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:10:50.084662 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a66b86-d9f5-452b-b3f2-a8f3bb2ff032 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:10:50.084932 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a66b86-d9f5-452b-b3f2-a8f3bb2ff032 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:10:50.086730 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d1a66b86-d9f5-452b-b3f2-a8f3bb2ff032 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:10:50.096460 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a66b86-d9f5-452b-b3f2-a8f3bb2ff032 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:10:50.097901 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a66b86-d9f5-452b-b3f2-a8f3bb2ff032 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:10:50.106685 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1026/2067] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:10:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:10:52.010695 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=31 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:11:00.103059 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=6 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:11:01.984399 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293126bb-ba32-4ce7-926e-aab964342f46 service nova] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:11:01.995810 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293126bb-ba32-4ce7-926e-aab964342f46 service nova] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:11:02.001272 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c141b5-17d9-4035-9c5e-e8e6c55e69b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:11:02.001863 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c141b5-17d9-4035-9c5e-e8e6c55e69b9 service nova] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:11:02.002140 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c141b5-17d9-4035-9c5e-e8e6c55e69b9 service nova] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:11:02.002461 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c141b5-17d9-4035-9c5e-e8e6c55e69b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:11:02.004168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-b5c141b5-17d9-4035-9c5e-e8e6c55e69b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:11:02.062647 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c141b5-17d9-4035-9c5e-e8e6c55e69b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:11:02.064255 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c141b5-17d9-4035-9c5e-e8e6c55e69b9 service nova] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:11:02.073794 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1042/2068] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:11:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:11:12.069191 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=28 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:11:14.755604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01834b-2ceb-4e9f-a4fd-f3e1ec81019d None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:11:14.756095 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01834b-2ceb-4e9f-a4fd-f3e1ec81019d None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:11:14.756193 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f01834b-2ceb-4e9f-a4fd-f3e1ec81019d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:11:14.775658 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-9f01834b-2ceb-4e9f-a4fd-f3e1ec81019d None None] MFA Rules not processed for user `4b6a101d23df4b62ba7e0c35d28d2a3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:11:14.777488 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9f01834b-2ceb-4e9f-a4fd-f3e1ec81019d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:11:14.784303 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1027/2069] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:11:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:11:20.346909 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c141b5-17d9-4035-9c5e-e8e6c55e69b9 service nova] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:11:20.358292 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c141b5-17d9-4035-9c5e-e8e6c55e69b9 service nova] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:11:20.365286 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27056f08-0d12-4796-adfc-7bd4740de15f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:11:20.365912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27056f08-0d12-4796-adfc-7bd4740de15f service nova] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:11:20.366188 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27056f08-0d12-4796-adfc-7bd4740de15f service nova] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:11:20.366560 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27056f08-0d12-4796-adfc-7bd4740de15f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:11:20.368287 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-27056f08-0d12-4796-adfc-7bd4740de15f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:11:20.381756 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27056f08-0d12-4796-adfc-7bd4740de15f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:11:20.383009 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27056f08-0d12-4796-adfc-7bd4740de15f service nova] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:11:20.395050 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1043/2070] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:11:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:11:24.770064 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:11:29.538173 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01834b-2ceb-4e9f-a4fd-f3e1ec81019d None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:11:29.552041 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f01834b-2ceb-4e9f-a4fd-f3e1ec81019d None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:11:29.557831 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c48088c-92be-41d2-8dd1-d8bf0ca80c59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:11:29.558476 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48088c-92be-41d2-8dd1-d8bf0ca80c59 service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:11:29.558755 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48088c-92be-41d2-8dd1-d8bf0ca80c59 service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:11:29.559023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48088c-92be-41d2-8dd1-d8bf0ca80c59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:11:29.561199 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-1c48088c-92be-41d2-8dd1-d8bf0ca80c59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:11:29.574843 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c48088c-92be-41d2-8dd1-d8bf0ca80c59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:11:29.576486 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c48088c-92be-41d2-8dd1-d8bf0ca80c59 service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:11:29.602354 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1028/2071] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:11:29 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:11:30.390299 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=6 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:11:37.010216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ded1f8-a69e-492a-99b2-b03c87f9f120 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:11:37.010855 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ded1f8-a69e-492a-99b2-b03c87f9f120 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:11:37.010948 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ded1f8-a69e-492a-99b2-b03c87f9f120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:11:37.031295 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-64ded1f8-a69e-492a-99b2-b03c87f9f120 None None] MFA Rules not processed for user `ff5cfe4d31614e2db9059c69116297cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:11:37.032499 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-64ded1f8-a69e-492a-99b2-b03c87f9f120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:11:37.039682 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1044/2072] 10.222.0.13 () {58 vars in 967 bytes} [Sun Oct 23 20:11:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:11:37.094479 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c48088c-92be-41d2-8dd1-d8bf0ca80c59 service nova] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:11:37.106883 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c48088c-92be-41d2-8dd1-d8bf0ca80c59 service nova] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:11:37.112143 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f259398-014c-4bd5-8e15-38fb251cf82d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15f603d1e7f54751aa79317e2ea00a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:11:37.112865 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f259398-014c-4bd5-8e15-38fb251cf82d service swift] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:11:37.113211 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f259398-014c-4bd5-8e15-38fb251cf82d service swift] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:11:37.113550 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f259398-014c-4bd5-8e15-38fb251cf82d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:11:37.122639 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f259398-014c-4bd5-8e15-38fb251cf82d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:11:37.124725 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f259398-014c-4bd5-8e15-38fb251cf82d service swift] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:11:37.134078 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1029/2073] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Oct 23 20:11:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 20:11:37.470323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ded1f8-a69e-492a-99b2-b03c87f9f120 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:11:37.480349 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ded1f8-a69e-492a-99b2-b03c87f9f120 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:11:37.484079 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3605d-1666-416e-b668-c672cc6b02ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:11:37.484711 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3605d-1666-416e-b668-c672cc6b02ea service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:11:37.485020 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3605d-1666-416e-b668-c672cc6b02ea service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:11:37.485294 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb3605d-1666-416e-b668-c672cc6b02ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:11:37.487044 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-2cb3605d-1666-416e-b668-c672cc6b02ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:11:37.548596 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb3605d-1666-416e-b668-c672cc6b02ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:11:37.550018 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb3605d-1666-416e-b668-c672cc6b02ea service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:11:37.560330 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1045/2074] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:11:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:11:40.628900 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f259398-014c-4bd5-8e15-38fb251cf82d service swift] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:11:40.630392 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-6f259398-014c-4bd5-8e15-38fb251cf82d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:11:40.705661 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f259398-014c-4bd5-8e15-38fb251cf82d service swift] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:11:40.710764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab39527-ce1d-4b61-93a2-690f18e43597 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312489c49a304ca9aacebd9843a4b65f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:11:40.711384 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab39527-ce1d-4b61-93a2-690f18e43597 service placement] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:11:40.711673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab39527-ce1d-4b61-93a2-690f18e43597 service placement] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:11:40.711953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab39527-ce1d-4b61-93a2-690f18e43597 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:11:40.713689 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-bab39527-ce1d-4b61-93a2-690f18e43597 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:11:40.725504 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab39527-ce1d-4b61-93a2-690f18e43597 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:11:40.727120 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab39527-ce1d-4b61-93a2-690f18e43597 service placement] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:11:40.737522 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1030/2075] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:11:40 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:11:44.077952 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3605d-1666-416e-b668-c672cc6b02ea service neutron] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:11:44.079628 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-2cb3605d-1666-416e-b668-c672cc6b02ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:11:44.092590 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb3605d-1666-416e-b668-c672cc6b02ea service neutron] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:11:44.097624 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c5dbf-2cc8-42a3-8d46-ccbfbb0c34db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:11:44.098311 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c5dbf-2cc8-42a3-8d46-ccbfbb0c34db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:11:44.098600 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c5dbf-2cc8-42a3-8d46-ccbfbb0c34db admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:11:44.098876 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92c5dbf-2cc8-42a3-8d46-ccbfbb0c34db admin admin] PATH_INFO: `/v3/users/7d0a9c2670794fb4b134e261411fd9cb` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:11:44.100004 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92c5dbf-2cc8-42a3-8d46-ccbfbb0c34db admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d0a9c2670794fb4b134e261411fd9cb)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:11:44.101927 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92c5dbf-2cc8-42a3-8d46-ccbfbb0c34db admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:11:44.156813 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-d92c5dbf-2cc8-42a3-8d46-ccbfbb0c34db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d0a9c2670794fb4b134e261411fd9cb'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:11:44.170321 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-d92c5dbf-2cc8-42a3-8d46-ccbfbb0c34db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d0a9c2670794fb4b134e261411fd9cb'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:11:44.177565 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-d92c5dbf-2cc8-42a3-8d46-ccbfbb0c34db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d0a9c2670794fb4b134e261411fd9cb'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:11:44.186737 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-d92c5dbf-2cc8-42a3-8d46-ccbfbb0c34db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d0a9c2670794fb4b134e261411fd9cb'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:11:44.187427 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-d92c5dbf-2cc8-42a3-8d46-ccbfbb0c34db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d0a9c2670794fb4b134e261411fd9cb'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:11:44.195094 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1046/2076] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:11:44 2022] DELETE /identity/v3/users/7d0a9c2670794fb4b134e261411fd9cb => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:11:44.378569 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab39527-ce1d-4b61-93a2-690f18e43597 service placement] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:11:44.389879 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab39527-ce1d-4b61-93a2-690f18e43597 service placement] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:11:44.449806 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e09dea-1c8d-47e3-9c25-d99cc6a81a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:11:44.450467 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e09dea-1c8d-47e3-9c25-d99cc6a81a2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:11:44.450744 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e09dea-1c8d-47e3-9c25-d99cc6a81a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:11:44.451016 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e09dea-1c8d-47e3-9c25-d99cc6a81a2e admin admin] PATH_INFO: `/v3/projects/6b4261a7b9e642e5ba063a4790296b66` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:11:44.452148 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e09dea-1c8d-47e3-9c25-d99cc6a81a2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b4261a7b9e642e5ba063a4790296b66)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:11:44.454240 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e09dea-1c8d-47e3-9c25-d99cc6a81a2e admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:11:44.503757 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-82e09dea-1c8d-47e3-9c25-d99cc6a81a2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b4261a7b9e642e5ba063a4790296b66', 'request_id': 'req-82e09dea-1c8d-47e3-9c25-d99cc6a81a2e'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:11:44.511310 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-82e09dea-1c8d-47e3-9c25-d99cc6a81a2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b4261a7b9e642e5ba063a4790296b66', 'request_id': 'req-82e09dea-1c8d-47e3-9c25-d99cc6a81a2e'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:11:44.517243 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-82e09dea-1c8d-47e3-9c25-d99cc6a81a2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b4261a7b9e642e5ba063a4790296b66', 'request_id': 'req-82e09dea-1c8d-47e3-9c25-d99cc6a81a2e'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:11:44.518631 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:11:44.532017 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-82e09dea-1c8d-47e3-9c25-d99cc6a81a2e admin admin] The token cache is being invalidate because project 6b4261a7b9e642e5ba063a4790296b66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:11:44.532320 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-82e09dea-1c8d-47e3-9c25-d99cc6a81a2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:11:44.532977 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-82e09dea-1c8d-47e3-9c25-d99cc6a81a2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:11:44.534512 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1031/2077] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:11:44 2022] DELETE /identity/v3/projects/6b4261a7b9e642e5ba063a4790296b66 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:11:44.538620 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c5dbf-2cc8-42a3-8d46-ccbfbb0c34db admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:11:44.540090 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-d92c5dbf-2cc8-42a3-8d46-ccbfbb0c34db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:11:44.554663 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92c5dbf-2cc8-42a3-8d46-ccbfbb0c34db admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:11:44.611262 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b1d4fe-a9a6-404c-a385-5fe0a61fecf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:11:44.611893 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1d4fe-a9a6-404c-a385-5fe0a61fecf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:11:44.612176 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1d4fe-a9a6-404c-a385-5fe0a61fecf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:11:44.612454 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b1d4fe-a9a6-404c-a385-5fe0a61fecf3 admin admin] PATH_INFO: `/v3/users/25cd9cc7253a4932ad013f1188995332` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:11:44.613574 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b1d4fe-a9a6-404c-a385-5fe0a61fecf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25cd9cc7253a4932ad013f1188995332)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:11:44.615552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b1d4fe-a9a6-404c-a385-5fe0a61fecf3 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:11:44.617013 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:11:44.662364 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-89b1d4fe-a9a6-404c-a385-5fe0a61fecf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25cd9cc7253a4932ad013f1188995332'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:11:44.674086 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-89b1d4fe-a9a6-404c-a385-5fe0a61fecf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25cd9cc7253a4932ad013f1188995332'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:11:44.680968 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-89b1d4fe-a9a6-404c-a385-5fe0a61fecf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25cd9cc7253a4932ad013f1188995332'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:11:44.688040 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-89b1d4fe-a9a6-404c-a385-5fe0a61fecf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25cd9cc7253a4932ad013f1188995332'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:11:44.688668 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-89b1d4fe-a9a6-404c-a385-5fe0a61fecf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25cd9cc7253a4932ad013f1188995332'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:11:44.694648 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1047/2078] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:11:44 2022] DELETE /identity/v3/users/25cd9cc7253a4932ad013f1188995332 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:11:45.116660 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e09dea-1c8d-47e3-9c25-d99cc6a81a2e admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:11:45.118401 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-82e09dea-1c8d-47e3-9c25-d99cc6a81a2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:11:45.133648 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e09dea-1c8d-47e3-9c25-d99cc6a81a2e admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:11:45.199696 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8014c-ece6-474d-b55d-fc824ad8cb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:11:45.200358 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8014c-ece6-474d-b55d-fc824ad8cb52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:11:45.200635 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8014c-ece6-474d-b55d-fc824ad8cb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:11:45.200906 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8014c-ece6-474d-b55d-fc824ad8cb52 admin admin] PATH_INFO: `/v3/projects/6f612c9285bd4cd9b00a0092f350d1d8` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:11:45.202016 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8014c-ece6-474d-b55d-fc824ad8cb52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f612c9285bd4cd9b00a0092f350d1d8)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:11:45.203984 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8014c-ece6-474d-b55d-fc824ad8cb52 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:11:45.253286 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-49c8014c-ece6-474d-b55d-fc824ad8cb52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f612c9285bd4cd9b00a0092f350d1d8', 'request_id': 'req-49c8014c-ece6-474d-b55d-fc824ad8cb52'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:11:45.261851 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-49c8014c-ece6-474d-b55d-fc824ad8cb52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f612c9285bd4cd9b00a0092f350d1d8', 'request_id': 'req-49c8014c-ece6-474d-b55d-fc824ad8cb52'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:11:45.268136 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-49c8014c-ece6-474d-b55d-fc824ad8cb52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f612c9285bd4cd9b00a0092f350d1d8', 'request_id': 'req-49c8014c-ece6-474d-b55d-fc824ad8cb52'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:11:45.279431 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-49c8014c-ece6-474d-b55d-fc824ad8cb52 admin admin] The token cache is being invalidate because project 6f612c9285bd4cd9b00a0092f350d1d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:11:45.279723 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-49c8014c-ece6-474d-b55d-fc824ad8cb52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:11:45.280413 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-49c8014c-ece6-474d-b55d-fc824ad8cb52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:11:45.281924 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1032/2079] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:11:45 2022] DELETE /identity/v3/projects/6f612c9285bd4cd9b00a0092f350d1d8 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:11:50.772488 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18db2aa-0119-4bc8-b4b5-b66c31ceecbe None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:11:50.773041 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18db2aa-0119-4bc8-b4b5-b66c31ceecbe None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:11:50.773041 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18db2aa-0119-4bc8-b4b5-b66c31ceecbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:11:50.793064 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-b18db2aa-0119-4bc8-b4b5-b66c31ceecbe None None] MFA Rules not processed for user `7cef68ee160348dea427e6c46b775e2d`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:11:50.794295 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-b18db2aa-0119-4bc8-b4b5-b66c31ceecbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:11:50.852290 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1048/2080] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:11:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:11:51.167653 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8014c-ece6-474d-b55d-fc824ad8cb52 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:11:51.169371 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-49c8014c-ece6-474d-b55d-fc824ad8cb52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:11:51.225999 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8014c-ece6-474d-b55d-fc824ad8cb52 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:11:51.229588 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad992f7-243a-4fc1-aa1e-480a96df4d6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:11:51.230191 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad992f7-243a-4fc1-aa1e-480a96df4d6d service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:11:51.230516 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad992f7-243a-4fc1-aa1e-480a96df4d6d service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:11:51.230788 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad992f7-243a-4fc1-aa1e-480a96df4d6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:11:51.232518 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-6ad992f7-243a-4fc1-aa1e-480a96df4d6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:11:51.241709 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad992f7-243a-4fc1-aa1e-480a96df4d6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:11:51.243152 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad992f7-243a-4fc1-aa1e-480a96df4d6d service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:11:51.251995 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1033/2081] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:11:51 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:11:51.561019 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18db2aa-0119-4bc8-b4b5-b66c31ceecbe None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:11:51.562450 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-b18db2aa-0119-4bc8-b4b5-b66c31ceecbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:11:51.645519 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18db2aa-0119-4bc8-b4b5-b66c31ceecbe None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:11:51.650497 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9377dc-83fb-4729-9726-96254e858060 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312489c49a304ca9aacebd9843a4b65f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:11:51.651225 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9377dc-83fb-4729-9726-96254e858060 service placement] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:11:51.651505 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9377dc-83fb-4729-9726-96254e858060 service placement] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:11:51.651812 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9377dc-83fb-4729-9726-96254e858060 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:11:51.653546 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-fc9377dc-83fb-4729-9726-96254e858060 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:11:51.665117 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9377dc-83fb-4729-9726-96254e858060 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:11:51.667295 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9377dc-83fb-4729-9726-96254e858060 service placement] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:11:51.676997 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1049/2082] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:11:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:11:59.781517 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad992f7-243a-4fc1-aa1e-480a96df4d6d service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:11:59.783110 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-6ad992f7-243a-4fc1-aa1e-480a96df4d6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:11:59.817337 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:11:59.867822 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad992f7-243a-4fc1-aa1e-480a96df4d6d service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:11:59.873418 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52745559-8023-481a-ab84-1da44a28dbcf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:11:59.874094 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52745559-8023-481a-ab84-1da44a28dbcf service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:11:59.874460 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52745559-8023-481a-ab84-1da44a28dbcf service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:11:59.874820 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52745559-8023-481a-ab84-1da44a28dbcf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:11:59.876706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-52745559-8023-481a-ab84-1da44a28dbcf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:11:59.889292 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52745559-8023-481a-ab84-1da44a28dbcf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:11:59.890916 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52745559-8023-481a-ab84-1da44a28dbcf service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:11:59.902697 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1034/2083] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:11:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:12:01.672870 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=83,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:12:09.898138 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=22 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:12:29.964839 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef9ddd-2999-4eb3-ba04-9bf9a6227fbd None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:12:29.965396 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef9ddd-2999-4eb3-ba04-9bf9a6227fbd None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:12:29.965396 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef9ddd-2999-4eb3-ba04-9bf9a6227fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:12:29.986461 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-84ef9ddd-2999-4eb3-ba04-9bf9a6227fbd None None] MFA Rules not processed for user `7aca00ea597140c5be97a6a4fa33f9c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:12:29.987666 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-84ef9ddd-2999-4eb3-ba04-9bf9a6227fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:12:30.045215 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1050/2084] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:12:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:12:30.361979 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52745559-8023-481a-ab84-1da44a28dbcf service nova] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:12:30.371956 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52745559-8023-481a-ab84-1da44a28dbcf service nova] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:12:30.378775 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e121d0db-0458-4f0d-b033-22f4dd1af19d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:12:30.379318 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121d0db-0458-4f0d-b033-22f4dd1af19d service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:12:30.379591 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121d0db-0458-4f0d-b033-22f4dd1af19d service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:12:30.379998 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e121d0db-0458-4f0d-b033-22f4dd1af19d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:12:30.381748 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e121d0db-0458-4f0d-b033-22f4dd1af19d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:12:30.393144 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e121d0db-0458-4f0d-b033-22f4dd1af19d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:12:30.394596 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e121d0db-0458-4f0d-b033-22f4dd1af19d service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:12:30.403559 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1035/2085] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:12:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:12:40.037301 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=25 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:12:40.399025 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=6 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:12:41.327773 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef9ddd-2999-4eb3-ba04-9bf9a6227fbd None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:12:41.339295 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef9ddd-2999-4eb3-ba04-9bf9a6227fbd None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:12:41.343463 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63429cf5-cc74-43bc-9c95-92fc3df3b4c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:12:41.344097 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63429cf5-cc74-43bc-9c95-92fc3df3b4c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:12:41.344358 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63429cf5-cc74-43bc-9c95-92fc3df3b4c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:12:41.344628 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63429cf5-cc74-43bc-9c95-92fc3df3b4c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:12:41.346787 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-63429cf5-cc74-43bc-9c95-92fc3df3b4c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:12:41.410717 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63429cf5-cc74-43bc-9c95-92fc3df3b4c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:12:41.412159 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63429cf5-cc74-43bc-9c95-92fc3df3b4c1 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:12:41.423202 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1051/2086] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:12:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:12:42.010391 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a559209-5014-40c2-977d-3e390cee5489 None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:12:42.010966 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a559209-5014-40c2-977d-3e390cee5489 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:12:42.011075 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a559209-5014-40c2-977d-3e390cee5489 None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:12:42.012678 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1036/2087] 10.222.0.13 () {54 vars in 921 bytes} [Sun Oct 23 20:12:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:12:42.021013 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed5571-0369-436d-89f4-c4b2ff1b873f None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:12:42.021331 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed5571-0369-436d-89f4-c4b2ff1b873f None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:12:42.022199 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ed5571-0369-436d-89f4-c4b2ff1b873f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:12:42.045689 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-76ed5571-0369-436d-89f4-c4b2ff1b873f None None] MFA Rules not processed for user `ba8ff3413bc549d29486d1c724a1f383`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:12:42.047439 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-76ed5571-0369-436d-89f4-c4b2ff1b873f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:12:42.058059 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1052/2088] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Oct 23 20:12:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:12:42.073593 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a559209-5014-40c2-977d-3e390cee5489 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:12:42.075602 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4a559209-5014-40c2-977d-3e390cee5489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:12:42.090808 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a559209-5014-40c2-977d-3e390cee5489 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:12:42.095423 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a62c2a-a7bb-46dc-82f9-7534a86876a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:12:42.096196 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a62c2a-a7bb-46dc-82f9-7534a86876a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:12:42.096612 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a62c2a-a7bb-46dc-82f9-7534a86876a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:12:42.097151 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a62c2a-a7bb-46dc-82f9-7534a86876a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:12:42.106099 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a62c2a-a7bb-46dc-82f9-7534a86876a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:12:42.107841 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a62c2a-a7bb-46dc-82f9-7534a86876a1 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:12:42.117979 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1037/2089] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:12:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:12:42.228536 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0810e-23b6-45b0-9d80-058364d0c647 None None] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:12:42.228888 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0810e-23b6-45b0-9d80-058364d0c647 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:12:42.229082 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa0810e-23b6-45b0-9d80-058364d0c647 None None] PATH_INFO: `/` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:12:42.230823 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1053/2090] 10.222.0.13 () {54 vars in 921 bytes} [Sun Oct 23 20:12:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:12:42.237184 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19feb087-82b4-4ba8-b573-d18bf5b7b262 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:12:42.238627 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19feb087-82b4-4ba8-b573-d18bf5b7b262 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:12:42.239074 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19feb087-82b4-4ba8-b573-d18bf5b7b262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:12:42.277396 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-19feb087-82b4-4ba8-b573-d18bf5b7b262 None None] MFA Rules not processed for user `0b3d512304254ffcaa8dad90abf1f9c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:12:42.279086 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-19feb087-82b4-4ba8-b573-d18bf5b7b262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:12:42.294359 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1038/2091] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Oct 23 20:12:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:12:42.308490 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0810e-23b6-45b0-9d80-058364d0c647 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:12:42.324606 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa0810e-23b6-45b0-9d80-058364d0c647 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:12:42.329118 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc963c7-18f3-4848-8c1c-d4db4b98a492 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:12:42.330504 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc963c7-18f3-4848-8c1c-d4db4b98a492 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:12:42.331028 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc963c7-18f3-4848-8c1c-d4db4b98a492 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:12:42.331747 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc963c7-18f3-4848-8c1c-d4db4b98a492 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:12:42.351264 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc963c7-18f3-4848-8c1c-d4db4b98a492 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:12:42.352740 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc963c7-18f3-4848-8c1c-d4db4b98a492 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:12:42.387356 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1054/2092] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:12:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:12:45.239663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19feb087-82b4-4ba8-b573-d18bf5b7b262 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:12:45.241382 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-19feb087-82b4-4ba8-b573-d18bf5b7b262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:12:45.264079 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19feb087-82b4-4ba8-b573-d18bf5b7b262 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:12:45.270631 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc390d89-2067-457c-9034-1fe9d2fee451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:12:45.271499 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc390d89-2067-457c-9034-1fe9d2fee451 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:12:45.271909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc390d89-2067-457c-9034-1fe9d2fee451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:12:45.272256 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc390d89-2067-457c-9034-1fe9d2fee451 admin admin] PATH_INFO: `/v3/users/4b6a101d23df4b62ba7e0c35d28d2a3c` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:12:45.273602 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc390d89-2067-457c-9034-1fe9d2fee451 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b6a101d23df4b62ba7e0c35d28d2a3c)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:12:45.276467 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc390d89-2067-457c-9034-1fe9d2fee451 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:12:45.330136 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-cc390d89-2067-457c-9034-1fe9d2fee451 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b6a101d23df4b62ba7e0c35d28d2a3c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:12:45.345635 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-cc390d89-2067-457c-9034-1fe9d2fee451 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b6a101d23df4b62ba7e0c35d28d2a3c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:12:45.355154 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-cc390d89-2067-457c-9034-1fe9d2fee451 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b6a101d23df4b62ba7e0c35d28d2a3c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:12:45.356288 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-cc390d89-2067-457c-9034-1fe9d2fee451 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b6a101d23df4b62ba7e0c35d28d2a3c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:12:45.364272 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-cc390d89-2067-457c-9034-1fe9d2fee451 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b6a101d23df4b62ba7e0c35d28d2a3c'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:12:45.370846 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1039/2093] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:12:45 2022] DELETE /identity/v3/users/4b6a101d23df4b62ba7e0c35d28d2a3c => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:12:45.554841 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc963c7-18f3-4848-8c1c-d4db4b98a492 service neutron] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:12:45.567586 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc963c7-18f3-4848-8c1c-d4db4b98a492 service neutron] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:12:45.637920 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf31a09c-9ed6-4106-8ae7-f8f4b93ee23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:12:45.637920 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31a09c-9ed6-4106-8ae7-f8f4b93ee23f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:12:45.637920 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31a09c-9ed6-4106-8ae7-f8f4b93ee23f admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:12:45.637920 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf31a09c-9ed6-4106-8ae7-f8f4b93ee23f admin admin] PATH_INFO: `/v3/projects/e41c21bcf4ca457391486c08565b2aae` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:12:45.639683 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf31a09c-9ed6-4106-8ae7-f8f4b93ee23f admin admin] RBAC: Authorizing `identity:delete_project(project_id=e41c21bcf4ca457391486c08565b2aae)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:12:45.641789 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf31a09c-9ed6-4106-8ae7-f8f4b93ee23f admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:12:45.692602 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-cf31a09c-9ed6-4106-8ae7-f8f4b93ee23f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e41c21bcf4ca457391486c08565b2aae', 'request_id': 'req-cf31a09c-9ed6-4106-8ae7-f8f4b93ee23f'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:12:45.701113 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-cf31a09c-9ed6-4106-8ae7-f8f4b93ee23f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e41c21bcf4ca457391486c08565b2aae', 'request_id': 'req-cf31a09c-9ed6-4106-8ae7-f8f4b93ee23f'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:12:45.707095 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-cf31a09c-9ed6-4106-8ae7-f8f4b93ee23f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e41c21bcf4ca457391486c08565b2aae', 'request_id': 'req-cf31a09c-9ed6-4106-8ae7-f8f4b93ee23f'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:12:45.719736 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-cf31a09c-9ed6-4106-8ae7-f8f4b93ee23f admin admin] The token cache is being invalidate because project e41c21bcf4ca457391486c08565b2aae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:12:45.720077 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-cf31a09c-9ed6-4106-8ae7-f8f4b93ee23f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:12:45.720677 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-cf31a09c-9ed6-4106-8ae7-f8f4b93ee23f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:12:45.722175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1055/2094] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:12:45 2022] DELETE /identity/v3/projects/e41c21bcf4ca457391486c08565b2aae => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:12:45.727028 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847f174-2109-4ae1-951d-37fb65999989 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:12:45.727313 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847f174-2109-4ae1-951d-37fb65999989 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:12:45.727586 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847f174-2109-4ae1-951d-37fb65999989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:12:45.748117 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-e847f174-2109-4ae1-951d-37fb65999989 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:12:45.749314 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e847f174-2109-4ae1-951d-37fb65999989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:12:45.810570 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1040/2095] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:12:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:12:45.819053 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8187d-05a1-4495-822a-25f8e73a9558 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:12:45.819343 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8187d-05a1-4495-822a-25f8e73a9558 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:12:45.819611 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df8187d-05a1-4495-822a-25f8e73a9558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:12:45.829048 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:12:45.843239 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-7df8187d-05a1-4495-822a-25f8e73a9558 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:12:45.844473 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-7df8187d-05a1-4495-822a-25f8e73a9558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:12:45.854383 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1056/2096] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:12:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:12:45.858969 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847f174-2109-4ae1-951d-37fb65999989 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:12:45.868696 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847f174-2109-4ae1-951d-37fb65999989 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:12:45.873800 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c409b98-3a84-40d2-a6a6-962a24c0c9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:12:45.874432 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c409b98-3a84-40d2-a6a6-962a24c0c9c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:12:45.874712 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c409b98-3a84-40d2-a6a6-962a24c0c9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:12:45.874985 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c409b98-3a84-40d2-a6a6-962a24c0c9c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:12:45.875634 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c409b98-3a84-40d2-a6a6-962a24c0c9c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:12:45.875932 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c409b98-3a84-40d2-a6a6-962a24c0c9c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:12:45.877520 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c409b98-3a84-40d2-a6a6-962a24c0c9c2 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:12:45.890381 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1041/2097] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:12:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:12:45.894865 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8187d-05a1-4495-822a-25f8e73a9558 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:12:45.904273 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df8187d-05a1-4495-822a-25f8e73a9558 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:12:45.909066 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914cc58c-2fd9-4836-a7fb-ffd62944d766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:12:45.909672 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914cc58c-2fd9-4836-a7fb-ffd62944d766 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:12:45.909983 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914cc58c-2fd9-4836-a7fb-ffd62944d766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:12:45.910269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914cc58c-2fd9-4836-a7fb-ffd62944d766 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:12:45.910958 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914cc58c-2fd9-4836-a7fb-ffd62944d766 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:12:45.912903 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914cc58c-2fd9-4836-a7fb-ffd62944d766 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:12:45.943529 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1057/2098] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:12:45 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:12:45.944825 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c409b98-3a84-40d2-a6a6-962a24c0c9c2 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:12:45.955531 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c409b98-3a84-40d2-a6a6-962a24c0c9c2 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:12:45.998294 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:12:46.025700 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14f1b0d-b528-43e6-bf0e-49bab6875e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:12:46.026357 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f1b0d-b528-43e6-bf0e-49bab6875e62 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:12:46.026645 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f1b0d-b528-43e6-bf0e-49bab6875e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:12:46.026921 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f1b0d-b528-43e6-bf0e-49bab6875e62 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:12:46.027567 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14f1b0d-b528-43e6-bf0e-49bab6875e62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:12:46.029574 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14f1b0d-b528-43e6-bf0e-49bab6875e62 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:12:46.056460 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1042/2099] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:12:45 2022] POST /identity/v3/users => generated 293 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:12:46.057975 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914cc58c-2fd9-4836-a7fb-ffd62944d766 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:12:46.068591 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914cc58c-2fd9-4836-a7fb-ffd62944d766 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:12:46.073548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf07a36-22ca-4299-973e-a3e5cf8531c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:12:46.074152 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf07a36-22ca-4299-973e-a3e5cf8531c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:12:46.074467 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf07a36-22ca-4299-973e-a3e5cf8531c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:12:46.074741 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf07a36-22ca-4299-973e-a3e5cf8531c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:12:46.075426 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf07a36-22ca-4299-973e-a3e5cf8531c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:12:46.077025 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf07a36-22ca-4299-973e-a3e5cf8531c7 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:12:46.087308 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1058/2100] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:12:46.091926 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14f1b0d-b528-43e6-bf0e-49bab6875e62 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:12:46.102094 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14f1b0d-b528-43e6-bf0e-49bab6875e62 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:12:46.107630 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a481f6d-e90b-4416-9556-ad0f7b488913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:12:46.108237 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a481f6d-e90b-4416-9556-ad0f7b488913 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:12:46.108523 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a481f6d-e90b-4416-9556-ad0f7b488913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:12:46.108797 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a481f6d-e90b-4416-9556-ad0f7b488913 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:12:46.109446 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a481f6d-e90b-4416-9556-ad0f7b488913 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:12:46.111124 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a481f6d-e90b-4416-9556-ad0f7b488913 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:12:46.122282 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1043/2101] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:12:46.126528 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf07a36-22ca-4299-973e-a3e5cf8531c7 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:12:46.136998 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf07a36-22ca-4299-973e-a3e5cf8531c7 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:12:46.141719 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17e6ab-3764-4286-bb71-0a17b8239cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:12:46.142479 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17e6ab-3764-4286-bb71-0a17b8239cdb admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:12:46.142753 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17e6ab-3764-4286-bb71-0a17b8239cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:12:46.143024 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca17e6ab-3764-4286-bb71-0a17b8239cdb admin admin] PATH_INFO: `/v3/projects/71f456c92d544d0c81bcb2fe937188ef/users/10bbbf111f714b108435b58c5b6c2747/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:12:46.163383 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca17e6ab-3764-4286-bb71-0a17b8239cdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=71f456c92d544d0c81bcb2fe937188ef, user_id=10bbbf111f714b108435b58c5b6c2747, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:12:46.166148 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca17e6ab-3764-4286-bb71-0a17b8239cdb admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:12:46.183357 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1059/2102] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:12:46 2022] PUT /identity/v3/projects/71f456c92d544d0c81bcb2fe937188ef/users/10bbbf111f714b108435b58c5b6c2747/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:12:46.188238 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6979f98-4e97-4e8b-a6b5-c103a7c5b6ba None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:12:46.188530 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6979f98-4e97-4e8b-a6b5-c103a7c5b6ba None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:12:46.188807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6979f98-4e97-4e8b-a6b5-c103a7c5b6ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:12:46.210576 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-d6979f98-4e97-4e8b-a6b5-c103a7c5b6ba None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:12:46.211790 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d6979f98-4e97-4e8b-a6b5-c103a7c5b6ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:12:46.275815 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1044/2103] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:12:46.307720 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17e6ab-3764-4286-bb71-0a17b8239cdb admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:12:46.309905 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-ca17e6ab-3764-4286-bb71-0a17b8239cdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:12:46.370978 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca17e6ab-3764-4286-bb71-0a17b8239cdb admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:12:46.374798 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3aeb23-8f3e-4a4a-89b6-d5ef8c08fce1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:12:46.375459 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3aeb23-8f3e-4a4a-89b6-d5ef8c08fce1 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:12:46.375742 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3aeb23-8f3e-4a4a-89b6-d5ef8c08fce1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:12:46.376016 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3aeb23-8f3e-4a4a-89b6-d5ef8c08fce1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:12:46.386533 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3aeb23-8f3e-4a4a-89b6-d5ef8c08fce1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:12:46.388000 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3aeb23-8f3e-4a4a-89b6-d5ef8c08fce1 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:12:46.398850 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1060/2104] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:12:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:12:51.301597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7baee08-85ed-4e78-b9a1-fd3093bb5858 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:12:51.302068 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7baee08-85ed-4e78-b9a1-fd3093bb5858 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:12:51.302228 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7baee08-85ed-4e78-b9a1-fd3093bb5858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:12:51.334556 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-c7baee08-85ed-4e78-b9a1-fd3093bb5858 None None] MFA Rules not processed for user `10bbbf111f714b108435b58c5b6c2747`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:12:51.335799 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c7baee08-85ed-4e78-b9a1-fd3093bb5858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:12:51.408184 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1045/2105] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:12:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:12:51.422792 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3aeb23-8f3e-4a4a-89b6-d5ef8c08fce1 service neutron] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:12:51.424425 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-ec3aeb23-8f3e-4a4a-89b6-d5ef8c08fce1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:12:51.511077 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3aeb23-8f3e-4a4a-89b6-d5ef8c08fce1 service neutron] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:12:51.516619 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc00d1-03d4-4711-ba22-99e1240382ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:12:51.517287 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc00d1-03d4-4711-ba22-99e1240382ef service nova] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:12:51.517517 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc00d1-03d4-4711-ba22-99e1240382ef service nova] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:12:51.517790 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dc00d1-03d4-4711-ba22-99e1240382ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:12:51.526798 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dc00d1-03d4-4711-ba22-99e1240382ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:12:51.528403 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23dc00d1-03d4-4711-ba22-99e1240382ef service nova] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:12:51.537188 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1061/2106] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:12:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:12:52.936866 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227ef20-4ec7-4ff0-84e7-a118dd0a9c11 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:12:52.937341 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227ef20-4ec7-4ff0-84e7-a118dd0a9c11 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:12:52.937442 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e227ef20-4ec7-4ff0-84e7-a118dd0a9c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:12:52.956380 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-e227ef20-4ec7-4ff0-84e7-a118dd0a9c11 None None] MFA Rules not processed for user `10bbbf111f714b108435b58c5b6c2747`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:12:52.957562 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e227ef20-4ec7-4ff0-84e7-a118dd0a9c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:12:52.964435 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1046/2107] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:12:52 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:13:01.533270 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=85,keystone:INSERT=2 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:13:02.951059 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=78,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:13:24.648578 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc00d1-03d4-4711-ba22-99e1240382ef service nova] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:13:24.649986 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-23dc00d1-03d4-4711-ba22-99e1240382ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:13:24.789793 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dc00d1-03d4-4711-ba22-99e1240382ef service nova] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:13:24.794798 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab967b54-8b24-457b-9760-3243b5297623 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312489c49a304ca9aacebd9843a4b65f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:13:24.795405 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab967b54-8b24-457b-9760-3243b5297623 service placement] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:13:24.795681 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab967b54-8b24-457b-9760-3243b5297623 service placement] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:13:24.795950 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab967b54-8b24-457b-9760-3243b5297623 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:13:24.797651 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-ab967b54-8b24-457b-9760-3243b5297623 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:13:24.808612 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab967b54-8b24-457b-9760-3243b5297623 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:13:24.810157 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab967b54-8b24-457b-9760-3243b5297623 service placement] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:13:24.820244 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1062/2108] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:13:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:13:34.816158 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=63 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:13:44.535821 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40113982-f804-4f98-ba67-cf361996da01 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:13:44.536385 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40113982-f804-4f98-ba67-cf361996da01 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:13:44.536491 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40113982-f804-4f98-ba67-cf361996da01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:13:44.566347 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-40113982-f804-4f98-ba67-cf361996da01 None None] MFA Rules not processed for user `ff5cfe4d31614e2db9059c69116297cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:13:44.567580 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-40113982-f804-4f98-ba67-cf361996da01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:13:44.629217 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1047/2109] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Oct 23 20:13:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:13:44.674985 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab967b54-8b24-457b-9760-3243b5297623 service placement] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:13:44.676379 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-ab967b54-8b24-457b-9760-3243b5297623 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:13:44.747629 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab967b54-8b24-457b-9760-3243b5297623 service placement] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:13:44.753378 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140dd421-f4fc-4c1b-8dea-98d52ca50c60 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15f603d1e7f54751aa79317e2ea00a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:13:44.754011 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140dd421-f4fc-4c1b-8dea-98d52ca50c60 service swift] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:13:44.754332 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140dd421-f4fc-4c1b-8dea-98d52ca50c60 service swift] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:13:44.754604 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140dd421-f4fc-4c1b-8dea-98d52ca50c60 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:13:44.762637 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140dd421-f4fc-4c1b-8dea-98d52ca50c60 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:13:44.764199 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140dd421-f4fc-4c1b-8dea-98d52ca50c60 service swift] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:13:44.772500 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1063/2110] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Oct 23 20:13:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 20:13:54.621872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=32 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:13:54.769299 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=32 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:13:59.821112 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc6d76-6c90-48b1-b649-524a75c31c5f None None] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:13:59.821610 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc6d76-6c90-48b1-b649-524a75c31c5f None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:13:59.821941 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc6d76-6c90-48b1-b649-524a75c31c5f None None] PATH_INFO: `/` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:13:59.823454 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1048/2111] 10.222.0.13 () {54 vars in 921 bytes} [Sun Oct 23 20:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 20:13:59.828722 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf25a8a-5793-4cdd-b9ea-31c499fe8854 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:13:59.829215 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf25a8a-5793-4cdd-b9ea-31c499fe8854 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:13:59.829322 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf25a8a-5793-4cdd-b9ea-31c499fe8854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:13:59.872212 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-ccf25a8a-5793-4cdd-b9ea-31c499fe8854 None None] MFA Rules not processed for user `0b3d512304254ffcaa8dad90abf1f9c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:13:59.873426 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-ccf25a8a-5793-4cdd-b9ea-31c499fe8854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:13:59.892166 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1064/2112] 10.222.0.13 () {58 vars in 1018 bytes} [Sun Oct 23 20:13:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:13:59.904217 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc6d76-6c90-48b1-b649-524a75c31c5f None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:13:59.905904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d2dc6d76-6c90-48b1-b649-524a75c31c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:13:59.966528 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc6d76-6c90-48b1-b649-524a75c31c5f None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:13:59.970028 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c6951-2de7-4897-b598-8674ebb3a6e2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '021216f90d7f4ff6b6eb7a1d86399d99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:13:59.970660 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c6951-2de7-4897-b598-8674ebb3a6e2 service glance] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:13:59.970933 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c6951-2de7-4897-b598-8674ebb3a6e2 service glance] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:13:59.971199 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c6951-2de7-4897-b598-8674ebb3a6e2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:13:59.981483 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c6951-2de7-4897-b598-8674ebb3a6e2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:13:59.982874 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c6951-2de7-4897-b598-8674ebb3a6e2 service glance] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:13:59.993360 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1049/2113] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:13:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:14:01.793609 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d3c6b-a49c-4161-9767-9235305ba8a6 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:14:01.794129 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d3c6b-a49c-4161-9767-9235305ba8a6 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:14:01.794129 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60d3c6b-a49c-4161-9767-9235305ba8a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:14:01.814048 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-c60d3c6b-a49c-4161-9767-9235305ba8a6 None None] MFA Rules not processed for user `ff5cfe4d31614e2db9059c69116297cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:14:01.815284 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-c60d3c6b-a49c-4161-9767-9235305ba8a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:14:01.822540 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1065/2114] 10.222.0.13 () {58 vars in 1009 bytes} [Sun Oct 23 20:14:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:14:01.834756 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c6951-2de7-4897-b598-8674ebb3a6e2 service glance] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:14:01.845764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c6951-2de7-4897-b598-8674ebb3a6e2 service glance] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:14:01.851573 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f6581-c680-4990-a228-5ac67a62ba67 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15f603d1e7f54751aa79317e2ea00a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:14:01.852110 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f6581-c680-4990-a228-5ac67a62ba67 service swift] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:14:01.852388 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f6581-c680-4990-a228-5ac67a62ba67 service swift] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:14:01.852694 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f6581-c680-4990-a228-5ac67a62ba67 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:14:01.861029 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8f6581-c680-4990-a228-5ac67a62ba67 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:14:01.863237 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8f6581-c680-4990-a228-5ac67a62ba67 service swift] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:14:01.872821 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1050/2115] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Oct 23 20:14:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 20:14:11.808783 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=19 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:14:11.869627 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=32 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:14:24.688810 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18641d5-db4b-4cf9-b377-4aef1946c6a8 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:14:24.689446 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18641d5-db4b-4cf9-b377-4aef1946c6a8 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:14:24.689446 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18641d5-db4b-4cf9-b377-4aef1946c6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:14:24.708923 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-a18641d5-db4b-4cf9-b377-4aef1946c6a8 None None] MFA Rules not processed for user `ff5cfe4d31614e2db9059c69116297cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:14:24.710107 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-a18641d5-db4b-4cf9-b377-4aef1946c6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:14:24.717311 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1066/2116] 10.222.0.13 () {58 vars in 967 bytes} [Sun Oct 23 20:14:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:14:24.729365 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f6581-c680-4990-a228-5ac67a62ba67 service swift] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:14:24.739805 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f6581-c680-4990-a228-5ac67a62ba67 service swift] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:14:24.745327 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29566b47-5215-455f-a16a-83a4af18ca10 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '15f603d1e7f54751aa79317e2ea00a31', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:14:24.745928 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29566b47-5215-455f-a16a-83a4af18ca10 service swift] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:14:24.746209 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29566b47-5215-455f-a16a-83a4af18ca10 service swift] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:14:24.746521 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29566b47-5215-455f-a16a-83a4af18ca10 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:14:24.754776 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29566b47-5215-455f-a16a-83a4af18ca10 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:14:24.756340 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29566b47-5215-455f-a16a-83a4af18ca10 service swift] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:14:24.765321 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1051/2117] 10.222.0.13 () {60 vars in 1384 bytes} [Sun Oct 23 20:14:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 20:14:34.703739 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:14:34.761718 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=6 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:14:37.591292 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18641d5-db4b-4cf9-b377-4aef1946c6a8 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:14:37.592805 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-a18641d5-db4b-4cf9-b377-4aef1946c6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:14:37.626712 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18641d5-db4b-4cf9-b377-4aef1946c6a8 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:14:37.630293 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd6697-317f-4c6a-894c-ab6d18834459 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:14:37.630899 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd6697-317f-4c6a-894c-ab6d18834459 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:14:37.631178 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd6697-317f-4c6a-894c-ab6d18834459 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:14:37.631450 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cd6697-317f-4c6a-894c-ab6d18834459 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:14:37.633152 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-11cd6697-317f-4c6a-894c-ab6d18834459 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:14:37.690280 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cd6697-317f-4c6a-894c-ab6d18834459 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:14:37.691709 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11cd6697-317f-4c6a-894c-ab6d18834459 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:14:37.700709 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1067/2118] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:14:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:14:42.555694 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29566b47-5215-455f-a16a-83a4af18ca10 service swift] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:14:42.565807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29566b47-5215-455f-a16a-83a4af18ca10 service swift] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:14:42.569471 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981ba25-daf2-4e17-a08b-dfe28abcd5d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:14:42.570095 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981ba25-daf2-4e17-a08b-dfe28abcd5d4 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:14:42.570423 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981ba25-daf2-4e17-a08b-dfe28abcd5d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:14:42.570702 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981ba25-daf2-4e17-a08b-dfe28abcd5d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:14:42.572425 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-8981ba25-daf2-4e17-a08b-dfe28abcd5d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:14:42.607126 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8981ba25-daf2-4e17-a08b-dfe28abcd5d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:14:42.608500 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8981ba25-daf2-4e17-a08b-dfe28abcd5d4 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:14:42.626924 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1052/2119] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:14:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:14:45.721228 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd6697-317f-4c6a-894c-ab6d18834459 service neutron] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:14:45.733873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cd6697-317f-4c6a-894c-ab6d18834459 service neutron] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:14:45.739662 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585119fa-c636-46a5-a496-fc9bc4bbf8a1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312489c49a304ca9aacebd9843a4b65f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:14:45.740313 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585119fa-c636-46a5-a496-fc9bc4bbf8a1 service placement] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:14:45.740587 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585119fa-c636-46a5-a496-fc9bc4bbf8a1 service placement] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:14:45.740858 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585119fa-c636-46a5-a496-fc9bc4bbf8a1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:14:45.742745 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-585119fa-c636-46a5-a496-fc9bc4bbf8a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:14:45.760680 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585119fa-c636-46a5-a496-fc9bc4bbf8a1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:14:45.762706 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585119fa-c636-46a5-a496-fc9bc4bbf8a1 service placement] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:14:45.786446 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1068/2120] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:14:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:14:47.466615 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99178f10-3fe2-47c3-b496-3120ca46361f None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:14:47.468112 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99178f10-3fe2-47c3-b496-3120ca46361f None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:14:47.468389 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99178f10-3fe2-47c3-b496-3120ca46361f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:14:47.491001 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-99178f10-3fe2-47c3-b496-3120ca46361f None None] MFA Rules not processed for user `10bbbf111f714b108435b58c5b6c2747`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:14:47.492565 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-99178f10-3fe2-47c3-b496-3120ca46361f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:14:47.500262 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1053/2121] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:14:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:14:50.144188 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585119fa-c636-46a5-a496-fc9bc4bbf8a1 service placement] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:14:50.145723 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-585119fa-c636-46a5-a496-fc9bc4bbf8a1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:14:50.167543 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585119fa-c636-46a5-a496-fc9bc4bbf8a1 service placement] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:14:50.175152 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac8368-a626-4a9f-b8ab-e275dca2796f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:14:50.176085 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac8368-a626-4a9f-b8ab-e275dca2796f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:14:50.176595 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac8368-a626-4a9f-b8ab-e275dca2796f admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:14:50.177093 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac8368-a626-4a9f-b8ab-e275dca2796f admin admin] PATH_INFO: `/v3/users/21fac7ecf4f2435483df150292a0052b` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:14:50.178537 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac8368-a626-4a9f-b8ab-e275dca2796f admin admin] RBAC: Authorizing `identity:delete_user(user_id=21fac7ecf4f2435483df150292a0052b)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:14:50.181939 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac8368-a626-4a9f-b8ab-e275dca2796f admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:14:50.240890 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-b5ac8368-a626-4a9f-b8ab-e275dca2796f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21fac7ecf4f2435483df150292a0052b'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:14:50.254944 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-b5ac8368-a626-4a9f-b8ab-e275dca2796f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21fac7ecf4f2435483df150292a0052b'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:14:50.264424 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-b5ac8368-a626-4a9f-b8ab-e275dca2796f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21fac7ecf4f2435483df150292a0052b'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:14:50.277692 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-b5ac8368-a626-4a9f-b8ab-e275dca2796f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21fac7ecf4f2435483df150292a0052b'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:14:50.278555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-b5ac8368-a626-4a9f-b8ab-e275dca2796f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21fac7ecf4f2435483df150292a0052b'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:14:50.294380 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1069/2122] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:14:50 2022] DELETE /identity/v3/users/21fac7ecf4f2435483df150292a0052b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:14:50.577473 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99178f10-3fe2-47c3-b496-3120ca46361f None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:14:50.601630 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99178f10-3fe2-47c3-b496-3120ca46361f None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:14:50.691258 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9942194-f21a-4d89-a0d7-cc6095dd70b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:14:50.691904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9942194-f21a-4d89-a0d7-cc6095dd70b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:14:50.692230 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9942194-f21a-4d89-a0d7-cc6095dd70b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:14:50.692498 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9942194-f21a-4d89-a0d7-cc6095dd70b4 admin admin] PATH_INFO: `/v3/projects/f84aaa230cf14fd2b47b2076e43fcb73` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:14:50.693860 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9942194-f21a-4d89-a0d7-cc6095dd70b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f84aaa230cf14fd2b47b2076e43fcb73)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:14:50.695806 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9942194-f21a-4d89-a0d7-cc6095dd70b4 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:14:50.778206 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a9942194-f21a-4d89-a0d7-cc6095dd70b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f84aaa230cf14fd2b47b2076e43fcb73', 'request_id': 'req-a9942194-f21a-4d89-a0d7-cc6095dd70b4'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:14:50.789316 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a9942194-f21a-4d89-a0d7-cc6095dd70b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f84aaa230cf14fd2b47b2076e43fcb73', 'request_id': 'req-a9942194-f21a-4d89-a0d7-cc6095dd70b4'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:14:50.799752 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a9942194-f21a-4d89-a0d7-cc6095dd70b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f84aaa230cf14fd2b47b2076e43fcb73', 'request_id': 'req-a9942194-f21a-4d89-a0d7-cc6095dd70b4'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:14:50.815261 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a9942194-f21a-4d89-a0d7-cc6095dd70b4 admin admin] The token cache is being invalidate because project f84aaa230cf14fd2b47b2076e43fcb73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:14:50.815688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a9942194-f21a-4d89-a0d7-cc6095dd70b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:14:50.816473 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-a9942194-f21a-4d89-a0d7-cc6095dd70b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:14:50.818442 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1054/2123] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:14:50 2022] DELETE /identity/v3/projects/f84aaa230cf14fd2b47b2076e43fcb73 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:14:50.822078 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac8368-a626-4a9f-b8ab-e275dca2796f admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:14:50.823600 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-b5ac8368-a626-4a9f-b8ab-e275dca2796f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:14:50.848470 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac8368-a626-4a9f-b8ab-e275dca2796f admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:14:50.902791 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:14:50.933682 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb3b47-882e-4d5c-a7a8-6811a7e562fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:14:50.934239 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb3b47-882e-4d5c-a7a8-6811a7e562fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:14:50.934662 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb3b47-882e-4d5c-a7a8-6811a7e562fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:14:50.935038 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfb3b47-882e-4d5c-a7a8-6811a7e562fe admin admin] PATH_INFO: `/v3/users/916e488424c447e4ac51df6c46989499` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:14:50.936983 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfb3b47-882e-4d5c-a7a8-6811a7e562fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=916e488424c447e4ac51df6c46989499)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:14:50.939130 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfb3b47-882e-4d5c-a7a8-6811a7e562fe admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:14:51.021410 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ebfb3b47-882e-4d5c-a7a8-6811a7e562fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '916e488424c447e4ac51df6c46989499'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:14:51.035599 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ebfb3b47-882e-4d5c-a7a8-6811a7e562fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '916e488424c447e4ac51df6c46989499'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:14:51.046730 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ebfb3b47-882e-4d5c-a7a8-6811a7e562fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '916e488424c447e4ac51df6c46989499'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:14:51.062173 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ebfb3b47-882e-4d5c-a7a8-6811a7e562fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '916e488424c447e4ac51df6c46989499'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:14:51.063167 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ebfb3b47-882e-4d5c-a7a8-6811a7e562fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '916e488424c447e4ac51df6c46989499'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:14:51.070203 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1070/2124] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:14:50 2022] DELETE /identity/v3/users/916e488424c447e4ac51df6c46989499 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:14:51.255201 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9942194-f21a-4d89-a0d7-cc6095dd70b4 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:14:51.256791 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a9942194-f21a-4d89-a0d7-cc6095dd70b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:14:51.289309 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9942194-f21a-4d89-a0d7-cc6095dd70b4 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:14:51.343782 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:14:51.364010 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab20458-e7b5-4356-9cf8-7bcd4f4b949d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:14:51.364769 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab20458-e7b5-4356-9cf8-7bcd4f4b949d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:14:51.365139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab20458-e7b5-4356-9cf8-7bcd4f4b949d admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:14:51.365575 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab20458-e7b5-4356-9cf8-7bcd4f4b949d admin admin] PATH_INFO: `/v3/projects/b81466a34db6497a94e114345a37e183` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:14:51.366805 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab20458-e7b5-4356-9cf8-7bcd4f4b949d admin admin] RBAC: Authorizing `identity:delete_project(project_id=b81466a34db6497a94e114345a37e183)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:14:51.368734 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab20458-e7b5-4356-9cf8-7bcd4f4b949d admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:14:51.423247 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-eab20458-e7b5-4356-9cf8-7bcd4f4b949d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b81466a34db6497a94e114345a37e183', 'request_id': 'req-eab20458-e7b5-4356-9cf8-7bcd4f4b949d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:14:51.431258 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-eab20458-e7b5-4356-9cf8-7bcd4f4b949d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b81466a34db6497a94e114345a37e183', 'request_id': 'req-eab20458-e7b5-4356-9cf8-7bcd4f4b949d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:14:51.437185 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-eab20458-e7b5-4356-9cf8-7bcd4f4b949d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b81466a34db6497a94e114345a37e183', 'request_id': 'req-eab20458-e7b5-4356-9cf8-7bcd4f4b949d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:14:51.448677 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-eab20458-e7b5-4356-9cf8-7bcd4f4b949d admin admin] The token cache is being invalidate because project b81466a34db6497a94e114345a37e183 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:14:51.448969 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-eab20458-e7b5-4356-9cf8-7bcd4f4b949d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:14:51.449898 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-eab20458-e7b5-4356-9cf8-7bcd4f4b949d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:14:51.451520 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1055/2125] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:14:51 2022] DELETE /identity/v3/projects/b81466a34db6497a94e114345a37e183 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:14:56.174853 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb3b47-882e-4d5c-a7a8-6811a7e562fe admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:14:56.176286 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-ebfb3b47-882e-4d5c-a7a8-6811a7e562fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:14:56.233220 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfb3b47-882e-4d5c-a7a8-6811a7e562fe admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:14:56.280910 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb284e5-7c37-43b1-9fdc-5d89d1fa0c65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:14:56.281454 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb284e5-7c37-43b1-9fdc-5d89d1fa0c65 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:14:56.281735 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb284e5-7c37-43b1-9fdc-5d89d1fa0c65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:14:56.282010 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb284e5-7c37-43b1-9fdc-5d89d1fa0c65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:14:56.283764 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-1fb284e5-7c37-43b1-9fdc-5d89d1fa0c65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:14:56.311171 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:14:56.348732 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb284e5-7c37-43b1-9fdc-5d89d1fa0c65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:14:56.350159 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb284e5-7c37-43b1-9fdc-5d89d1fa0c65 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:14:56.359870 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1071/2126] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:14:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:14:59.298685 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab20458-e7b5-4356-9cf8-7bcd4f4b949d admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:14:59.300722 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-eab20458-e7b5-4356-9cf8-7bcd4f4b949d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:14:59.321599 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab20458-e7b5-4356-9cf8-7bcd4f4b949d admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:14:59.328114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce74cba-52d7-4165-8de7-db7622ce3a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:14:59.329030 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce74cba-52d7-4165-8de7-db7622ce3a36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:14:59.329400 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce74cba-52d7-4165-8de7-db7622ce3a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:14:59.329779 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce74cba-52d7-4165-8de7-db7622ce3a36 admin admin] PATH_INFO: `/v3/users/7aca00ea597140c5be97a6a4fa33f9c7` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:14:59.331094 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce74cba-52d7-4165-8de7-db7622ce3a36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7aca00ea597140c5be97a6a4fa33f9c7)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:14:59.333346 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce74cba-52d7-4165-8de7-db7622ce3a36 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:14:59.418690 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9ce74cba-52d7-4165-8de7-db7622ce3a36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7aca00ea597140c5be97a6a4fa33f9c7'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:14:59.431952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9ce74cba-52d7-4165-8de7-db7622ce3a36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7aca00ea597140c5be97a6a4fa33f9c7'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:14:59.447158 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9ce74cba-52d7-4165-8de7-db7622ce3a36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7aca00ea597140c5be97a6a4fa33f9c7'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:14:59.447788 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9ce74cba-52d7-4165-8de7-db7622ce3a36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7aca00ea597140c5be97a6a4fa33f9c7'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:14:59.468500 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9ce74cba-52d7-4165-8de7-db7622ce3a36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7aca00ea597140c5be97a6a4fa33f9c7'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:14:59.474919 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1056/2127] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:14:59 2022] DELETE /identity/v3/users/7aca00ea597140c5be97a6a4fa33f9c7 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:14:59.708666 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb284e5-7c37-43b1-9fdc-5d89d1fa0c65 service neutron] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:14:59.719035 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb284e5-7c37-43b1-9fdc-5d89d1fa0c65 service neutron] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:14:59.781222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c28144-9202-4dee-b487-3ae39b36139a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:14:59.781770 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c28144-9202-4dee-b487-3ae39b36139a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:14:59.782050 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c28144-9202-4dee-b487-3ae39b36139a admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:14:59.782355 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c28144-9202-4dee-b487-3ae39b36139a admin admin] PATH_INFO: `/v3/projects/5ebb0d7d16214912a585ca96cf6b2c6a` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:14:59.783490 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c28144-9202-4dee-b487-3ae39b36139a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ebb0d7d16214912a585ca96cf6b2c6a)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:14:59.785422 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c28144-9202-4dee-b487-3ae39b36139a admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:14:59.835275 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-28c28144-9202-4dee-b487-3ae39b36139a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ebb0d7d16214912a585ca96cf6b2c6a', 'request_id': 'req-28c28144-9202-4dee-b487-3ae39b36139a'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:14:59.843471 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-28c28144-9202-4dee-b487-3ae39b36139a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ebb0d7d16214912a585ca96cf6b2c6a', 'request_id': 'req-28c28144-9202-4dee-b487-3ae39b36139a'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:14:59.849805 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-28c28144-9202-4dee-b487-3ae39b36139a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ebb0d7d16214912a585ca96cf6b2c6a', 'request_id': 'req-28c28144-9202-4dee-b487-3ae39b36139a'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:14:59.862313 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-28c28144-9202-4dee-b487-3ae39b36139a admin admin] The token cache is being invalidate because project 5ebb0d7d16214912a585ca96cf6b2c6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:14:59.862313 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-28c28144-9202-4dee-b487-3ae39b36139a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:14:59.862899 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-28c28144-9202-4dee-b487-3ae39b36139a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:14:59.864389 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1072/2128] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:14:59 2022] DELETE /identity/v3/projects/5ebb0d7d16214912a585ca96cf6b2c6a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:14:59.869813 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b23d2-c0f7-4e45-b7bc-79c205b482af None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:14:59.870100 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b23d2-c0f7-4e45-b7bc-79c205b482af None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:14:59.870409 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b23d2-c0f7-4e45-b7bc-79c205b482af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:14:59.918541 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-009b23d2-c0f7-4e45-b7bc-79c205b482af None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:14:59.919749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-009b23d2-c0f7-4e45-b7bc-79c205b482af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:14:59.937516 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:14:59.982012 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1057/2129] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:14:59.991578 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b70ed-c815-4515-ab09-373f84e5d2b2 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:14:59.991862 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b70ed-c815-4515-ab09-373f84e5d2b2 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:14:59.992136 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181b70ed-c815-4515-ab09-373f84e5d2b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:00.012159 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-181b70ed-c815-4515-ab09-373f84e5d2b2 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:00.013380 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-181b70ed-c815-4515-ab09-373f84e5d2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:00.026204 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1073/2130] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:00.030309 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b23d2-c0f7-4e45-b7bc-79c205b482af None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:00.040139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009b23d2-c0f7-4e45-b7bc-79c205b482af None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:00.044603 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf9c3c-a996-41e5-ad1e-4a5190de68b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:00.045716 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf9c3c-a996-41e5-ad1e-4a5190de68b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:00.046011 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf9c3c-a996-41e5-ad1e-4a5190de68b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:00.046358 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bdf9c3c-a996-41e5-ad1e-4a5190de68b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:00.047039 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf9c3c-a996-41e5-ad1e-4a5190de68b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:15:00.047339 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf9c3c-a996-41e5-ad1e-4a5190de68b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:00.048940 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bdf9c3c-a996-41e5-ad1e-4a5190de68b9 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:00.061358 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1058/2131] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:15:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 20:15:00.065701 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b70ed-c815-4515-ab09-373f84e5d2b2 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:00.075815 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181b70ed-c815-4515-ab09-373f84e5d2b2 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:00.080678 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d21bd9f-3e92-4655-860a-23767f378fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:00.081353 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21bd9f-3e92-4655-860a-23767f378fa3 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:00.081724 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21bd9f-3e92-4655-860a-23767f378fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:00.082064 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21bd9f-3e92-4655-860a-23767f378fa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:00.082817 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d21bd9f-3e92-4655-860a-23767f378fa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:00.084802 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d21bd9f-3e92-4655-860a-23767f378fa3 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:00.113084 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1074/2132] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:15:00 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:15:00.117408 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf9c3c-a996-41e5-ad1e-4a5190de68b9 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:00.128760 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bdf9c3c-a996-41e5-ad1e-4a5190de68b9 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:00.192301 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979fbec5-d80a-4228-b2da-f1003dbc0291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:00.192903 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fbec5-d80a-4228-b2da-f1003dbc0291 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:00.193181 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fbec5-d80a-4228-b2da-f1003dbc0291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:00.193450 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fbec5-d80a-4228-b2da-f1003dbc0291 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:00.194118 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979fbec5-d80a-4228-b2da-f1003dbc0291 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:00.196349 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979fbec5-d80a-4228-b2da-f1003dbc0291 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:00.223906 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1059/2133] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:15:00 2022] POST /identity/v3/users => generated 288 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:15:00.225130 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d21bd9f-3e92-4655-860a-23767f378fa3 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:00.236224 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d21bd9f-3e92-4655-860a-23767f378fa3 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:00.240802 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75199d-507f-4fa1-9838-3c64bb711a81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:00.241398 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75199d-507f-4fa1-9838-3c64bb711a81 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:00.241675 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75199d-507f-4fa1-9838-3c64bb711a81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:00.241934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75199d-507f-4fa1-9838-3c64bb711a81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:00.242617 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df75199d-507f-4fa1-9838-3c64bb711a81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:00.244202 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df75199d-507f-4fa1-9838-3c64bb711a81 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:00.254380 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1075/2134] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:15:00.258331 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979fbec5-d80a-4228-b2da-f1003dbc0291 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:00.268613 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979fbec5-d80a-4228-b2da-f1003dbc0291 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:00.273299 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7224bc15-fbf4-4cc3-aad3-a66a4b80e754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:00.273896 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224bc15-fbf4-4cc3-aad3-a66a4b80e754 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:00.274177 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224bc15-fbf4-4cc3-aad3-a66a4b80e754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:00.274488 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7224bc15-fbf4-4cc3-aad3-a66a4b80e754 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:00.275165 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7224bc15-fbf4-4cc3-aad3-a66a4b80e754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:00.276752 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7224bc15-fbf4-4cc3-aad3-a66a4b80e754 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:00.288410 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1060/2135] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 20:15:00.292926 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75199d-507f-4fa1-9838-3c64bb711a81 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:00.303640 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75199d-507f-4fa1-9838-3c64bb711a81 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:00.309173 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe20459-52cb-471e-baa1-839072e4b9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:00.309883 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe20459-52cb-471e-baa1-839072e4b9e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:00.310170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe20459-52cb-471e-baa1-839072e4b9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:00.310478 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe20459-52cb-471e-baa1-839072e4b9e9 admin admin] PATH_INFO: `/v3/projects/0289c86cb5b947c0b0fb326895d6f77e/users/dbababb082d94ab485a2b343d00d913b/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:00.324285 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:00.328480 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe20459-52cb-471e-baa1-839072e4b9e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0289c86cb5b947c0b0fb326895d6f77e, user_id=dbababb082d94ab485a2b343d00d913b, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:00.332297 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe20459-52cb-471e-baa1-839072e4b9e9 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:00.350185 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1076/2136] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:15:00 2022] PUT /identity/v3/projects/0289c86cb5b947c0b0fb326895d6f77e/users/dbababb082d94ab485a2b343d00d913b/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:00.361166 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881f4b8-0abb-4bac-afc2-32b3b18be908 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:00.361736 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881f4b8-0abb-4bac-afc2-32b3b18be908 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:00.361921 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7881f4b8-0abb-4bac-afc2-32b3b18be908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:00.392484 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-7881f4b8-0abb-4bac-afc2-32b3b18be908 None None] MFA Rules not processed for user `dbababb082d94ab485a2b343d00d913b`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:00.393719 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7881f4b8-0abb-4bac-afc2-32b3b18be908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:00.463571 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1061/2137] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:00.475682 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe20459-52cb-471e-baa1-839072e4b9e9 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:00.477433 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-cbe20459-52cb-471e-baa1-839072e4b9e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:00.553484 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe20459-52cb-471e-baa1-839072e4b9e9 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:00.559039 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00036378-a7e4-4fa4-9102-07e57c644aec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:00.559653 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00036378-a7e4-4fa4-9102-07e57c644aec service nova] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:00.559930 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00036378-a7e4-4fa4-9102-07e57c644aec service nova] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:00.560206 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00036378-a7e4-4fa4-9102-07e57c644aec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:00.568174 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00036378-a7e4-4fa4-9102-07e57c644aec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:00.569749 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00036378-a7e4-4fa4-9102-07e57c644aec service nova] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:00.579320 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1077/2138] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:15:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:15:00.806096 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7881f4b8-0abb-4bac-afc2-32b3b18be908 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:00.830496 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:00.886277 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7881f4b8-0abb-4bac-afc2-32b3b18be908 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:00.891503 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590863c1-01a5-4db4-869f-651c51c5a81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:00.892148 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590863c1-01a5-4db4-869f-651c51c5a81b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:00.892428 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590863c1-01a5-4db4-869f-651c51c5a81b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:00.892701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590863c1-01a5-4db4-869f-651c51c5a81b admin admin] PATH_INFO: `/v3/users/dbababb082d94ab485a2b343d00d913b` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:00.894200 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590863c1-01a5-4db4-869f-651c51c5a81b admin admin] RBAC: Authorizing `identity:delete_user(user_id=dbababb082d94ab485a2b343d00d913b)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:00.895828 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590863c1-01a5-4db4-869f-651c51c5a81b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:00.954206 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-590863c1-01a5-4db4-869f-651c51c5a81b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dbababb082d94ab485a2b343d00d913b'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:00.970053 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-590863c1-01a5-4db4-869f-651c51c5a81b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dbababb082d94ab485a2b343d00d913b'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:00.980094 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-590863c1-01a5-4db4-869f-651c51c5a81b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dbababb082d94ab485a2b343d00d913b'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:00.981060 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-590863c1-01a5-4db4-869f-651c51c5a81b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dbababb082d94ab485a2b343d00d913b'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:00.990078 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-590863c1-01a5-4db4-869f-651c51c5a81b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dbababb082d94ab485a2b343d00d913b'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:00.997345 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1062/2139] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:15:00 2022] DELETE /identity/v3/users/dbababb082d94ab485a2b343d00d913b => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:15:01.002753 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167919c-78c0-48cc-9314-9a1ede3d8694 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:01.003730 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167919c-78c0-48cc-9314-9a1ede3d8694 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:01.004879 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e167919c-78c0-48cc-9314-9a1ede3d8694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:01.030228 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-e167919c-78c0-48cc-9314-9a1ede3d8694 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:01.031492 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-e167919c-78c0-48cc-9314-9a1ede3d8694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:01.095044 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1078/2140] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:01.105545 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590863c1-01a5-4db4-869f-651c51c5a81b admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:01.108219 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-590863c1-01a5-4db4-869f-651c51c5a81b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:01.171212 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590863c1-01a5-4db4-869f-651c51c5a81b admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:01.177119 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e2cf1-6eac-4da8-9f46-c7ca35a4b3e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:01.177863 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e2cf1-6eac-4da8-9f46-c7ca35a4b3e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:01.178148 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e2cf1-6eac-4da8-9f46-c7ca35a4b3e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:01.178446 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e2cf1-6eac-4da8-9f46-c7ca35a4b3e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:01.189294 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90e2cf1-6eac-4da8-9f46-c7ca35a4b3e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:01.190774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90e2cf1-6eac-4da8-9f46-c7ca35a4b3e2 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:01.201888 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1063/2141] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:15:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:15:01.477921 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e167919c-78c0-48cc-9314-9a1ede3d8694 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:01.479372 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-e167919c-78c0-48cc-9314-9a1ede3d8694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:01.491798 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e167919c-78c0-48cc-9314-9a1ede3d8694 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:01.497037 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f79f7-cd15-4145-a49b-4aa3b7500530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:01.497759 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f79f7-cd15-4145-a49b-4aa3b7500530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:01.498728 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f79f7-cd15-4145-a49b-4aa3b7500530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:01.499113 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f79f7-cd15-4145-a49b-4aa3b7500530 admin admin] PATH_INFO: `/v3/projects/0289c86cb5b947c0b0fb326895d6f77e` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:01.500398 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f79f7-cd15-4145-a49b-4aa3b7500530 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0289c86cb5b947c0b0fb326895d6f77e)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:01.503792 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f79f7-cd15-4145-a49b-4aa3b7500530 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:01.552423 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-781f79f7-cd15-4145-a49b-4aa3b7500530 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0289c86cb5b947c0b0fb326895d6f77e', 'request_id': 'req-781f79f7-cd15-4145-a49b-4aa3b7500530'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:01.560339 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-781f79f7-cd15-4145-a49b-4aa3b7500530 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0289c86cb5b947c0b0fb326895d6f77e', 'request_id': 'req-781f79f7-cd15-4145-a49b-4aa3b7500530'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:01.567717 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-781f79f7-cd15-4145-a49b-4aa3b7500530 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0289c86cb5b947c0b0fb326895d6f77e', 'request_id': 'req-781f79f7-cd15-4145-a49b-4aa3b7500530'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:01.576793 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:01.581848 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-781f79f7-cd15-4145-a49b-4aa3b7500530 admin admin] The token cache is being invalidate because project 0289c86cb5b947c0b0fb326895d6f77e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:15:01.582153 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-781f79f7-cd15-4145-a49b-4aa3b7500530 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:01.583304 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-781f79f7-cd15-4145-a49b-4aa3b7500530 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:01.585443 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1079/2142] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:15:01 2022] DELETE /identity/v3/projects/0289c86cb5b947c0b0fb326895d6f77e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:01.589947 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34dd87-23a1-4daa-8971-ea976acf41b1 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:01.590464 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34dd87-23a1-4daa-8971-ea976acf41b1 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:01.590698 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c34dd87-23a1-4daa-8971-ea976acf41b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:01.611693 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-3c34dd87-23a1-4daa-8971-ea976acf41b1 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:01.612912 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3c34dd87-23a1-4daa-8971-ea976acf41b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:01.647464 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:01.675315 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1064/2143] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:01.683272 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b8698-95f0-41c8-b927-c69a20834330 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:01.683556 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b8698-95f0-41c8-b927-c69a20834330 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:01.683833 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b8698-95f0-41c8-b927-c69a20834330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:01.704689 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-6d8b8698-95f0-41c8-b927-c69a20834330 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:01.705899 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-6d8b8698-95f0-41c8-b927-c69a20834330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:01.713881 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1080/2144] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:15:01.718783 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34dd87-23a1-4daa-8971-ea976acf41b1 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:01.728828 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c34dd87-23a1-4daa-8971-ea976acf41b1 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:01.733693 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b92af95-bc71-41d9-86e8-c4e0148568da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:01.734378 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92af95-bc71-41d9-86e8-c4e0148568da admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:01.734669 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92af95-bc71-41d9-86e8-c4e0148568da admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:01.734951 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b92af95-bc71-41d9-86e8-c4e0148568da admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:01.735612 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b92af95-bc71-41d9-86e8-c4e0148568da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:15:01.735921 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b92af95-bc71-41d9-86e8-c4e0148568da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:01.737539 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b92af95-bc71-41d9-86e8-c4e0148568da admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:01.750886 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1065/2145] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:15:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:15:01.755215 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b8698-95f0-41c8-b927-c69a20834330 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:01.765962 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b8698-95f0-41c8-b927-c69a20834330 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:01.771192 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cab962-f086-466d-97ca-19bf30155a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:01.772059 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cab962-f086-466d-97ca-19bf30155a9b admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:01.772420 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cab962-f086-466d-97ca-19bf30155a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:01.772772 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cab962-f086-466d-97ca-19bf30155a9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:01.773495 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cab962-f086-466d-97ca-19bf30155a9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:01.777194 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cab962-f086-466d-97ca-19bf30155a9b admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:01.807184 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1081/2146] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:15:01 2022] POST /identity/v3/projects => generated 401 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:15:01.808325 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b92af95-bc71-41d9-86e8-c4e0148568da admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:01.818859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b92af95-bc71-41d9-86e8-c4e0148568da admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:01.877891 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9fcdd2-2fea-4e3e-b0f9-c4889de98622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:01.878565 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fcdd2-2fea-4e3e-b0f9-c4889de98622 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:01.878925 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fcdd2-2fea-4e3e-b0f9-c4889de98622 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:01.879203 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9fcdd2-2fea-4e3e-b0f9-c4889de98622 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:01.879860 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9fcdd2-2fea-4e3e-b0f9-c4889de98622 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:01.881812 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9fcdd2-2fea-4e3e-b0f9-c4889de98622 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:01.903836 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1066/2147] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:15:01 2022] POST /identity/v3/users => generated 301 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:15:01.908273 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cab962-f086-466d-97ca-19bf30155a9b admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:01.918887 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cab962-f086-466d-97ca-19bf30155a9b admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:01.924161 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d5e33-5074-4174-bc4b-c2dde62971e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:01.924786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d5e33-5074-4174-bc4b-c2dde62971e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:01.925071 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d5e33-5074-4174-bc4b-c2dde62971e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:01.925376 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d5e33-5074-4174-bc4b-c2dde62971e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:01.926138 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96d5e33-5074-4174-bc4b-c2dde62971e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:01.927832 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96d5e33-5074-4174-bc4b-c2dde62971e5 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:01.939497 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1082/2148] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:15:01.943728 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9fcdd2-2fea-4e3e-b0f9-c4889de98622 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:01.955062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9fcdd2-2fea-4e3e-b0f9-c4889de98622 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:01.959752 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf05806d-8b27-42aa-82a0-70cc1d9d1788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:01.960349 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05806d-8b27-42aa-82a0-70cc1d9d1788 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:01.960628 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05806d-8b27-42aa-82a0-70cc1d9d1788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:01.960958 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf05806d-8b27-42aa-82a0-70cc1d9d1788 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:01.961601 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf05806d-8b27-42aa-82a0-70cc1d9d1788 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:01.963227 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf05806d-8b27-42aa-82a0-70cc1d9d1788 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:01.973787 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1067/2149] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:15:01.977901 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d5e33-5074-4174-bc4b-c2dde62971e5 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:01.987762 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d5e33-5074-4174-bc4b-c2dde62971e5 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:01.992328 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7a90c-9729-4b22-acdf-d02f69ec4be8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:01.992971 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7a90c-9729-4b22-acdf-d02f69ec4be8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:01.993252 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7a90c-9729-4b22-acdf-d02f69ec4be8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:01.993522 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7a90c-9729-4b22-acdf-d02f69ec4be8 admin admin] PATH_INFO: `/v3/projects/08f9697671e84342900418238b2b0d41/users/b7d7b581f7b44a7d890188c691582890/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:02.010761 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f7a90c-9729-4b22-acdf-d02f69ec4be8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08f9697671e84342900418238b2b0d41, user_id=b7d7b581f7b44a7d890188c691582890, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:02.014505 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f7a90c-9729-4b22-acdf-d02f69ec4be8 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:02.028395 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1083/2150] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:15:01 2022] PUT /identity/v3/projects/08f9697671e84342900418238b2b0d41/users/b7d7b581f7b44a7d890188c691582890/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:02.039896 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec158c0-725a-4b9e-b019-19304dc00133 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:02.040180 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec158c0-725a-4b9e-b019-19304dc00133 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:02.040450 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec158c0-725a-4b9e-b019-19304dc00133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:02.072907 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-0ec158c0-725a-4b9e-b019-19304dc00133 None None] MFA Rules not processed for user `b7d7b581f7b44a7d890188c691582890`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:02.074125 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-0ec158c0-725a-4b9e-b019-19304dc00133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:02.137282 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1068/2151] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:02 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:02.144446 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7a90c-9729-4b22-acdf-d02f69ec4be8 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:02.206873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7a90c-9729-4b22-acdf-d02f69ec4be8 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:02.211405 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564a37e-4f55-4a80-b96b-9bcd0a22c579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:02.212002 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564a37e-4f55-4a80-b96b-9bcd0a22c579 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:02.212280 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564a37e-4f55-4a80-b96b-9bcd0a22c579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:02.212551 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a564a37e-4f55-4a80-b96b-9bcd0a22c579 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:02.213182 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a564a37e-4f55-4a80-b96b-9bcd0a22c579 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:02.215104 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a564a37e-4f55-4a80-b96b-9bcd0a22c579 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:02.241011 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1084/2152] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:15:02 2022] POST /identity/v3/projects => generated 405 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:15:02.245107 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec158c0-725a-4b9e-b019-19304dc00133 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:02.256321 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec158c0-725a-4b9e-b019-19304dc00133 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:02.287999 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:02.314516 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ed7db-80af-4e22-962d-c4a12ad383a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:02.315154 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ed7db-80af-4e22-962d-c4a12ad383a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:02.315431 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ed7db-80af-4e22-962d-c4a12ad383a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:02.315728 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ed7db-80af-4e22-962d-c4a12ad383a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:02.316355 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6ed7db-80af-4e22-962d-c4a12ad383a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:02.318345 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6ed7db-80af-4e22-962d-c4a12ad383a0 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:02.339010 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1069/2153] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:15:02 2022] POST /identity/v3/users => generated 309 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:15:02.343038 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564a37e-4f55-4a80-b96b-9bcd0a22c579 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:02.353359 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a564a37e-4f55-4a80-b96b-9bcd0a22c579 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:02.357911 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67bcb63-d610-4377-bbf1-0a67368e750d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:02.359041 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67bcb63-d610-4377-bbf1-0a67368e750d admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:02.359318 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67bcb63-d610-4377-bbf1-0a67368e750d admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:02.359585 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67bcb63-d610-4377-bbf1-0a67368e750d admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:02.360221 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67bcb63-d610-4377-bbf1-0a67368e750d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:02.361910 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67bcb63-d610-4377-bbf1-0a67368e750d admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:02.372290 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1085/2154] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:15:02.377102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ed7db-80af-4e22-962d-c4a12ad383a0 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:02.386850 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ed7db-80af-4e22-962d-c4a12ad383a0 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:02.391466 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0456979e-669d-4e85-8c65-231c2bb47cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:02.392055 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456979e-669d-4e85-8c65-231c2bb47cdd admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:02.392332 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456979e-669d-4e85-8c65-231c2bb47cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:02.392603 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456979e-669d-4e85-8c65-231c2bb47cdd admin admin] PATH_INFO: `/v3/projects/09a4f2df4cb94ef3a2567c3867d29606/users/462d2e58b6924d2da68c6c415f2d9dcc/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:02.407321 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0456979e-669d-4e85-8c65-231c2bb47cdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=09a4f2df4cb94ef3a2567c3867d29606, user_id=462d2e58b6924d2da68c6c415f2d9dcc, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:02.410193 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0456979e-669d-4e85-8c65-231c2bb47cdd admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:02.423181 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1070/2155] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:15:02 2022] PUT /identity/v3/projects/09a4f2df4cb94ef3a2567c3867d29606/users/462d2e58b6924d2da68c6c415f2d9dcc/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:02.427206 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67bcb63-d610-4377-bbf1-0a67368e750d admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:02.437665 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67bcb63-d610-4377-bbf1-0a67368e750d admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:02.493216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b843fca8-7051-47e6-b7dd-2c3b515951b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:02.493854 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843fca8-7051-47e6-b7dd-2c3b515951b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:02.494134 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843fca8-7051-47e6-b7dd-2c3b515951b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:02.494447 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b843fca8-7051-47e6-b7dd-2c3b515951b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:02.495092 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b843fca8-7051-47e6-b7dd-2c3b515951b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:02.496685 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b843fca8-7051-47e6-b7dd-2c3b515951b4 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:02.506288 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1086/2156] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:15:02.510586 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0456979e-669d-4e85-8c65-231c2bb47cdd admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:02.520696 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0456979e-669d-4e85-8c65-231c2bb47cdd admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:02.525260 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a93ff5-a22a-4e21-bb66-273f35da769e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:02.525906 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a93ff5-a22a-4e21-bb66-273f35da769e admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:02.526190 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a93ff5-a22a-4e21-bb66-273f35da769e admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:02.526498 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a93ff5-a22a-4e21-bb66-273f35da769e admin admin] PATH_INFO: `/v3/projects/09a4f2df4cb94ef3a2567c3867d29606/users/462d2e58b6924d2da68c6c415f2d9dcc/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:02.528407 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a93ff5-a22a-4e21-bb66-273f35da769e admin admin] RBAC: Authorizing `identity:create_grant(project_id=09a4f2df4cb94ef3a2567c3867d29606, user_id=462d2e58b6924d2da68c6c415f2d9dcc, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:02.531588 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a93ff5-a22a-4e21-bb66-273f35da769e admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:02.541659 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1071/2157] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:15:02 2022] PUT /identity/v3/projects/09a4f2df4cb94ef3a2567c3867d29606/users/462d2e58b6924d2da68c6c415f2d9dcc/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:02.552320 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c775873-8037-4b21-bc1b-820dad6f015d None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:02.552607 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c775873-8037-4b21-bc1b-820dad6f015d None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:02.552877 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c775873-8037-4b21-bc1b-820dad6f015d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:02.560806 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:02.581556 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-7c775873-8037-4b21-bc1b-820dad6f015d None None] MFA Rules not processed for user `462d2e58b6924d2da68c6c415f2d9dcc`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:02.582792 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-7c775873-8037-4b21-bc1b-820dad6f015d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:02.650010 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1087/2158] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:02 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:02.661089 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a93ff5-a22a-4e21-bb66-273f35da769e admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:02.662815 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-00a93ff5-a22a-4e21-bb66-273f35da769e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:02.741056 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a93ff5-a22a-4e21-bb66-273f35da769e admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:02.744745 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b5cde5-2747-42dd-943c-b225ee6ab421 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:02.745342 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5cde5-2747-42dd-943c-b225ee6ab421 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:02.745622 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5cde5-2747-42dd-943c-b225ee6ab421 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:02.745895 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b5cde5-2747-42dd-943c-b225ee6ab421 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:02.754819 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b5cde5-2747-42dd-943c-b225ee6ab421 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:02.756238 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b5cde5-2747-42dd-943c-b225ee6ab421 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:02.765490 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1072/2159] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:15:02 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:15:02.901471 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c775873-8037-4b21-bc1b-820dad6f015d None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:02.961742 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c775873-8037-4b21-bc1b-820dad6f015d None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:02.966343 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4d8de-6d8e-492b-a2f4-c1b98dc1ac68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:02.966964 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4d8de-6d8e-492b-a2f4-c1b98dc1ac68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:02.967240 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4d8de-6d8e-492b-a2f4-c1b98dc1ac68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:02.968112 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4d8de-6d8e-492b-a2f4-c1b98dc1ac68 admin admin] PATH_INFO: `/v3/users/b7d7b581f7b44a7d890188c691582890` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:02.969219 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4d8de-6d8e-492b-a2f4-c1b98dc1ac68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7d7b581f7b44a7d890188c691582890)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:02.971184 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4d8de-6d8e-492b-a2f4-c1b98dc1ac68 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:03.018681 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-86b4d8de-6d8e-492b-a2f4-c1b98dc1ac68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7d7b581f7b44a7d890188c691582890'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:03.031703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-86b4d8de-6d8e-492b-a2f4-c1b98dc1ac68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7d7b581f7b44a7d890188c691582890'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:03.038587 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-86b4d8de-6d8e-492b-a2f4-c1b98dc1ac68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7d7b581f7b44a7d890188c691582890'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:03.046720 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-86b4d8de-6d8e-492b-a2f4-c1b98dc1ac68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7d7b581f7b44a7d890188c691582890'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:03.047477 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-86b4d8de-6d8e-492b-a2f4-c1b98dc1ac68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7d7b581f7b44a7d890188c691582890'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:03.054435 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1088/2160] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:15:02 2022] DELETE /identity/v3/users/b7d7b581f7b44a7d890188c691582890 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:03.058718 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054dea7-d231-4018-8db9-85313ecf297f None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:03.059030 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054dea7-d231-4018-8db9-85313ecf297f None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:03.059307 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1054dea7-d231-4018-8db9-85313ecf297f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:03.078826 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-1054dea7-d231-4018-8db9-85313ecf297f None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:03.080069 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-1054dea7-d231-4018-8db9-85313ecf297f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:03.137380 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1073/2161] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:03.147455 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4d8de-6d8e-492b-a2f4-c1b98dc1ac68 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:03.148946 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-86b4d8de-6d8e-492b-a2f4-c1b98dc1ac68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:03.160619 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:03.208030 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4d8de-6d8e-492b-a2f4-c1b98dc1ac68 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:03.211610 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0290cf2-e7ce-4f9f-8404-9a39abacba11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:03.212206 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0290cf2-e7ce-4f9f-8404-9a39abacba11 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:03.212481 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0290cf2-e7ce-4f9f-8404-9a39abacba11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:03.212746 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0290cf2-e7ce-4f9f-8404-9a39abacba11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:03.221986 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0290cf2-e7ce-4f9f-8404-9a39abacba11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:03.223432 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0290cf2-e7ce-4f9f-8404-9a39abacba11 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:03.232714 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1089/2162] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:15:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:15:03.495929 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1054dea7-d231-4018-8db9-85313ecf297f None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:03.497405 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-1054dea7-d231-4018-8db9-85313ecf297f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:03.510142 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1054dea7-d231-4018-8db9-85313ecf297f None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:03.515861 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e54c8-e667-4c2e-be33-0fe35c3b608f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:03.516495 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e54c8-e667-4c2e-be33-0fe35c3b608f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:03.516774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e54c8-e667-4c2e-be33-0fe35c3b608f admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:03.517064 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838e54c8-e667-4c2e-be33-0fe35c3b608f admin admin] PATH_INFO: `/v3/projects/08f9697671e84342900418238b2b0d41` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:03.518221 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e54c8-e667-4c2e-be33-0fe35c3b608f admin admin] RBAC: Authorizing `identity:delete_project(project_id=08f9697671e84342900418238b2b0d41)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:03.520133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838e54c8-e667-4c2e-be33-0fe35c3b608f admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:03.527087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:03.567732 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-838e54c8-e667-4c2e-be33-0fe35c3b608f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08f9697671e84342900418238b2b0d41', 'request_id': 'req-838e54c8-e667-4c2e-be33-0fe35c3b608f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:03.574841 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-838e54c8-e667-4c2e-be33-0fe35c3b608f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08f9697671e84342900418238b2b0d41', 'request_id': 'req-838e54c8-e667-4c2e-be33-0fe35c3b608f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:03.580917 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-838e54c8-e667-4c2e-be33-0fe35c3b608f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08f9697671e84342900418238b2b0d41', 'request_id': 'req-838e54c8-e667-4c2e-be33-0fe35c3b608f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:03.591753 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-838e54c8-e667-4c2e-be33-0fe35c3b608f admin admin] The token cache is being invalidate because project 08f9697671e84342900418238b2b0d41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:15:03.592039 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-838e54c8-e667-4c2e-be33-0fe35c3b608f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:03.592651 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-838e54c8-e667-4c2e-be33-0fe35c3b608f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:03.594229 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1074/2163] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:15:03 2022] DELETE /identity/v3/projects/08f9697671e84342900418238b2b0d41 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:03.598168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0290cf2-e7ce-4f9f-8404-9a39abacba11 service neutron] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:03.608536 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0290cf2-e7ce-4f9f-8404-9a39abacba11 service neutron] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:03.663331 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f2dea0-3c4a-463a-be02-db8775e234d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:03.663949 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2dea0-3c4a-463a-be02-db8775e234d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:03.664228 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2dea0-3c4a-463a-be02-db8775e234d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:03.664502 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f2dea0-3c4a-463a-be02-db8775e234d8 admin admin] PATH_INFO: `/v3/users/462d2e58b6924d2da68c6c415f2d9dcc` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:03.665612 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f2dea0-3c4a-463a-be02-db8775e234d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=462d2e58b6924d2da68c6c415f2d9dcc)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:03.667551 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f2dea0-3c4a-463a-be02-db8775e234d8 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:03.712681 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-02f2dea0-3c4a-463a-be02-db8775e234d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '462d2e58b6924d2da68c6c415f2d9dcc'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:03.724099 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-02f2dea0-3c4a-463a-be02-db8775e234d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '462d2e58b6924d2da68c6c415f2d9dcc'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:03.730800 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-02f2dea0-3c4a-463a-be02-db8775e234d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '462d2e58b6924d2da68c6c415f2d9dcc'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:03.738325 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-02f2dea0-3c4a-463a-be02-db8775e234d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '462d2e58b6924d2da68c6c415f2d9dcc'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:03.738938 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-02f2dea0-3c4a-463a-be02-db8775e234d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '462d2e58b6924d2da68c6c415f2d9dcc'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:03.744934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1090/2164] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:15:03 2022] DELETE /identity/v3/users/462d2e58b6924d2da68c6c415f2d9dcc => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:04.009647 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e54c8-e667-4c2e-be33-0fe35c3b608f admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:04.011186 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-838e54c8-e667-4c2e-be33-0fe35c3b608f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:04.027834 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838e54c8-e667-4c2e-be33-0fe35c3b608f admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:04.085885 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb947dd-0466-4146-8c2c-f27aac43566a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:04.086556 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb947dd-0466-4146-8c2c-f27aac43566a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:04.086840 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb947dd-0466-4146-8c2c-f27aac43566a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:04.087118 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb947dd-0466-4146-8c2c-f27aac43566a admin admin] PATH_INFO: `/v3/projects/09a4f2df4cb94ef3a2567c3867d29606` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:04.088233 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb947dd-0466-4146-8c2c-f27aac43566a admin admin] RBAC: Authorizing `identity:delete_project(project_id=09a4f2df4cb94ef3a2567c3867d29606)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:04.090140 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb947dd-0466-4146-8c2c-f27aac43566a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:04.136272 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-0cb947dd-0466-4146-8c2c-f27aac43566a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09a4f2df4cb94ef3a2567c3867d29606', 'request_id': 'req-0cb947dd-0466-4146-8c2c-f27aac43566a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:04.143381 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-0cb947dd-0466-4146-8c2c-f27aac43566a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09a4f2df4cb94ef3a2567c3867d29606', 'request_id': 'req-0cb947dd-0466-4146-8c2c-f27aac43566a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:04.149181 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-0cb947dd-0466-4146-8c2c-f27aac43566a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09a4f2df4cb94ef3a2567c3867d29606', 'request_id': 'req-0cb947dd-0466-4146-8c2c-f27aac43566a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:04.160599 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-0cb947dd-0466-4146-8c2c-f27aac43566a admin admin] The token cache is being invalidate because project 09a4f2df4cb94ef3a2567c3867d29606 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:15:04.160883 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-0cb947dd-0466-4146-8c2c-f27aac43566a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:04.161515 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-0cb947dd-0466-4146-8c2c-f27aac43566a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:04.163035 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1075/2165] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:15:04 2022] DELETE /identity/v3/projects/09a4f2df4cb94ef3a2567c3867d29606 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:04.167929 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17558f-bd9b-4ad6-b70d-acbea950e402 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:04.168383 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17558f-bd9b-4ad6-b70d-acbea950e402 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:04.168476 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b17558f-bd9b-4ad6-b70d-acbea950e402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:04.188193 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-0b17558f-bd9b-4ad6-b70d-acbea950e402 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:04.189389 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-0b17558f-bd9b-4ad6-b70d-acbea950e402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:04.214433 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:04.250320 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1091/2166] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:15:04.259619 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c5f34-e8b9-4db2-b7c4-588200464908 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:04.259849 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c5f34-e8b9-4db2-b7c4-588200464908 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:04.260125 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74c5f34-e8b9-4db2-b7c4-588200464908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:04.279802 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-f74c5f34-e8b9-4db2-b7c4-588200464908 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:04.281611 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f74c5f34-e8b9-4db2-b7c4-588200464908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:04.289774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1076/2167] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:04.293872 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17558f-bd9b-4ad6-b70d-acbea950e402 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:04.304110 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b17558f-bd9b-4ad6-b70d-acbea950e402 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:04.308886 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52ad84-0adf-4666-b0ca-5b597f4d4d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:04.309512 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52ad84-0adf-4666-b0ca-5b597f4d4d07 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:04.309788 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52ad84-0adf-4666-b0ca-5b597f4d4d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:04.310058 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52ad84-0adf-4666-b0ca-5b597f4d4d07 admin admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:04.310731 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52ad84-0adf-4666-b0ca-5b597f4d4d07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:15:04.311030 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52ad84-0adf-4666-b0ca-5b597f4d4d07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:04.312694 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52ad84-0adf-4666-b0ca-5b597f4d4d07 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:04.324170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1092/2168] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:15:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:15:04.329008 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c5f34-e8b9-4db2-b7c4-588200464908 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:04.337965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74c5f34-e8b9-4db2-b7c4-588200464908 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:04.343189 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc2d2df-64ce-44bf-b999-0870ed6f15c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:04.343778 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2d2df-64ce-44bf-b999-0870ed6f15c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:04.344053 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2d2df-64ce-44bf-b999-0870ed6f15c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:04.344326 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc2d2df-64ce-44bf-b999-0870ed6f15c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:04.344959 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc2d2df-64ce-44bf-b999-0870ed6f15c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:04.346875 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc2d2df-64ce-44bf-b999-0870ed6f15c7 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:04.362210 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:04.378893 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1077/2169] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:15:04 2022] POST /identity/v3/projects => generated 379 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:15:04.380089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52ad84-0adf-4666-b0ca-5b597f4d4d07 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:04.390471 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52ad84-0adf-4666-b0ca-5b597f4d4d07 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:04.445382 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc0e10-111e-4e04-96f7-02906a1b9377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:04.446012 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc0e10-111e-4e04-96f7-02906a1b9377 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:04.446318 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc0e10-111e-4e04-96f7-02906a1b9377 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:04.446594 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bc0e10-111e-4e04-96f7-02906a1b9377 admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:04.447225 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc0e10-111e-4e04-96f7-02906a1b9377 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:04.449134 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bc0e10-111e-4e04-96f7-02906a1b9377 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:04.471572 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1093/2170] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:15:04 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:15:04.475775 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc2d2df-64ce-44bf-b999-0870ed6f15c7 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:04.485954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc2d2df-64ce-44bf-b999-0870ed6f15c7 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:04.490535 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4936bc10-7f62-47e8-b382-8fa34c250fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:04.491126 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936bc10-7f62-47e8-b382-8fa34c250fa2 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:04.491405 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936bc10-7f62-47e8-b382-8fa34c250fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:04.491674 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4936bc10-7f62-47e8-b382-8fa34c250fa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:04.492312 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4936bc10-7f62-47e8-b382-8fa34c250fa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:04.493890 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4936bc10-7f62-47e8-b382-8fa34c250fa2 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:04.503970 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1078/2171] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:15:04.507791 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc0e10-111e-4e04-96f7-02906a1b9377 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:04.518468 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bc0e10-111e-4e04-96f7-02906a1b9377 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:04.523054 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d102e0c-ee87-4b27-872c-0e5b6547c55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:04.523652 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d102e0c-ee87-4b27-872c-0e5b6547c55c admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:04.523933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d102e0c-ee87-4b27-872c-0e5b6547c55c admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:04.524207 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d102e0c-ee87-4b27-872c-0e5b6547c55c admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:04.524852 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d102e0c-ee87-4b27-872c-0e5b6547c55c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:04.526519 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d102e0c-ee87-4b27-872c-0e5b6547c55c admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:04.537730 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1094/2172] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:15:04.541810 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4936bc10-7f62-47e8-b382-8fa34c250fa2 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:04.551625 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4936bc10-7f62-47e8-b382-8fa34c250fa2 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:04.556271 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e7323-6628-4f3d-9e0b-78dd145f1ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:04.556935 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e7323-6628-4f3d-9e0b-78dd145f1ec2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:04.557215 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e7323-6628-4f3d-9e0b-78dd145f1ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:04.557489 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3e7323-6628-4f3d-9e0b-78dd145f1ec2 admin admin] PATH_INFO: `/v3/projects/38c70ab7ca2044fca7d84782193ae8a3/users/f9dc549329e740feb28d4f04caa2958f/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:04.571965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e7323-6628-4f3d-9e0b-78dd145f1ec2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38c70ab7ca2044fca7d84782193ae8a3, user_id=f9dc549329e740feb28d4f04caa2958f, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:04.574925 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3e7323-6628-4f3d-9e0b-78dd145f1ec2 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:04.588077 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1079/2173] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:15:04 2022] PUT /identity/v3/projects/38c70ab7ca2044fca7d84782193ae8a3/users/f9dc549329e740feb28d4f04caa2958f/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:04.598907 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d35901-b37e-47b7-8cb6-6df72c3e2441 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:04.599227 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d35901-b37e-47b7-8cb6-6df72c3e2441 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:04.599498 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d35901-b37e-47b7-8cb6-6df72c3e2441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:04.627500 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-93d35901-b37e-47b7-8cb6-6df72c3e2441 None None] MFA Rules not processed for user `f9dc549329e740feb28d4f04caa2958f`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:04.628762 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-93d35901-b37e-47b7-8cb6-6df72c3e2441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:04.691199 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1095/2174] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:04.695602 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e7323-6628-4f3d-9e0b-78dd145f1ec2 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:04.756487 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3e7323-6628-4f3d-9e0b-78dd145f1ec2 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:04.761080 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f8abd8-63c4-4cb6-825f-3d249d81529d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:04.761678 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8abd8-63c4-4cb6-825f-3d249d81529d admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:04.761957 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8abd8-63c4-4cb6-825f-3d249d81529d admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:04.762230 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f8abd8-63c4-4cb6-825f-3d249d81529d admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:04.762932 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f8abd8-63c4-4cb6-825f-3d249d81529d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:04.765479 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f8abd8-63c4-4cb6-825f-3d249d81529d admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:04.791874 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1080/2175] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:15:04 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:15:04.795823 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d35901-b37e-47b7-8cb6-6df72c3e2441 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:04.806281 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d35901-b37e-47b7-8cb6-6df72c3e2441 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:04.831473 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:04.864117 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804d158-facb-4910-8bda-979781819569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:04.864716 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804d158-facb-4910-8bda-979781819569 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:04.864991 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804d158-facb-4910-8bda-979781819569 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:04.865260 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804d158-facb-4910-8bda-979781819569 admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:04.865879 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2804d158-facb-4910-8bda-979781819569 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:04.868244 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2804d158-facb-4910-8bda-979781819569 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:04.890396 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1096/2176] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:15:04 2022] POST /identity/v3/users => generated 297 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:15:04.894397 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f8abd8-63c4-4cb6-825f-3d249d81529d admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:04.904409 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f8abd8-63c4-4cb6-825f-3d249d81529d admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:04.909409 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e782297c-20e7-4524-84e1-f463cfff1332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:04.909994 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782297c-20e7-4524-84e1-f463cfff1332 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:04.910299 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782297c-20e7-4524-84e1-f463cfff1332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:04.910575 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782297c-20e7-4524-84e1-f463cfff1332 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:04.911207 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e782297c-20e7-4524-84e1-f463cfff1332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:04.912780 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e782297c-20e7-4524-84e1-f463cfff1332 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:04.923413 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1081/2177] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:15:04.927197 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804d158-facb-4910-8bda-979781819569 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:04.937530 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804d158-facb-4910-8bda-979781819569 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:04.942123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3a185-1056-498d-a5cd-575ddeb09551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:04.942860 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3a185-1056-498d-a5cd-575ddeb09551 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:04.943135 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3a185-1056-498d-a5cd-575ddeb09551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:04.943402 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3a185-1056-498d-a5cd-575ddeb09551 admin admin] PATH_INFO: `/v3/projects/7fffc43ffc2d404aa022a40fbfcdbc78/users/f57f9751c5954ef4ab103ae51dd9a7f2/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:04.958477 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd3a185-1056-498d-a5cd-575ddeb09551 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fffc43ffc2d404aa022a40fbfcdbc78, user_id=f57f9751c5954ef4ab103ae51dd9a7f2, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:04.961104 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd3a185-1056-498d-a5cd-575ddeb09551 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:04.973997 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1097/2178] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:15:04 2022] PUT /identity/v3/projects/7fffc43ffc2d404aa022a40fbfcdbc78/users/f57f9751c5954ef4ab103ae51dd9a7f2/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:04.977894 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e782297c-20e7-4524-84e1-f463cfff1332 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:04.987759 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e782297c-20e7-4524-84e1-f463cfff1332 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:05.043588 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac5859a-553e-4df9-882a-982493787d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:05.044181 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5859a-553e-4df9-882a-982493787d94 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:05.044391 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5859a-553e-4df9-882a-982493787d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:05.044661 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac5859a-553e-4df9-882a-982493787d94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:05.045295 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac5859a-553e-4df9-882a-982493787d94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:05.046904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac5859a-553e-4df9-882a-982493787d94 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:05.056386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1082/2179] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:15:05.060487 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3a185-1056-498d-a5cd-575ddeb09551 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:05.071339 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3a185-1056-498d-a5cd-575ddeb09551 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:05.076271 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de43507-29e1-45e2-bd94-ce43c364e1b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:05.077470 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de43507-29e1-45e2-bd94-ce43c364e1b0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:05.077750 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de43507-29e1-45e2-bd94-ce43c364e1b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:05.078023 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de43507-29e1-45e2-bd94-ce43c364e1b0 admin admin] PATH_INFO: `/v3/projects/7fffc43ffc2d404aa022a40fbfcdbc78/users/f57f9751c5954ef4ab103ae51dd9a7f2/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:05.080003 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de43507-29e1-45e2-bd94-ce43c364e1b0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fffc43ffc2d404aa022a40fbfcdbc78, user_id=f57f9751c5954ef4ab103ae51dd9a7f2, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:05.082731 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de43507-29e1-45e2-bd94-ce43c364e1b0 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:05.092867 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1098/2180] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:15:05 2022] PUT /identity/v3/projects/7fffc43ffc2d404aa022a40fbfcdbc78/users/f57f9751c5954ef4ab103ae51dd9a7f2/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:05.102892 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f341298-797b-4a08-b2e2-bed3a683b401 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:05.103180 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f341298-797b-4a08-b2e2-bed3a683b401 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:05.103456 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f341298-797b-4a08-b2e2-bed3a683b401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:05.131989 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-9f341298-797b-4a08-b2e2-bed3a683b401 None None] MFA Rules not processed for user `f57f9751c5954ef4ab103ae51dd9a7f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:05.133204 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9f341298-797b-4a08-b2e2-bed3a683b401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:05.151195 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:05.202861 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1083/2181] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:05 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:15:05.214071 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de43507-29e1-45e2-bd94-ce43c364e1b0 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:05.215802 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-1de43507-29e1-45e2-bd94-ce43c364e1b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:05.273013 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de43507-29e1-45e2-bd94-ce43c364e1b0 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:05.276709 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251e2dad-e628-46a7-8055-81bf385ffdee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:05.277307 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e2dad-e628-46a7-8055-81bf385ffdee service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:05.277587 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e2dad-e628-46a7-8055-81bf385ffdee service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:05.277860 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251e2dad-e628-46a7-8055-81bf385ffdee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:05.286712 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251e2dad-e628-46a7-8055-81bf385ffdee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:05.288136 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251e2dad-e628-46a7-8055-81bf385ffdee service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:05.297975 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1099/2182] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:15:05 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:15:05.373125 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f341298-797b-4a08-b2e2-bed3a683b401 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:05.383435 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f341298-797b-4a08-b2e2-bed3a683b401 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:05.387096 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3eae1e-27c4-4723-b67e-ea47d4d0bf95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:05.387693 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3eae1e-27c4-4723-b67e-ea47d4d0bf95 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:05.387967 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3eae1e-27c4-4723-b67e-ea47d4d0bf95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:05.388233 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3eae1e-27c4-4723-b67e-ea47d4d0bf95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:05.444446 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3eae1e-27c4-4723-b67e-ea47d4d0bf95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:05.445867 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3eae1e-27c4-4723-b67e-ea47d4d0bf95 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:05.454986 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1084/2183] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:15:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:15:15.293211 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=58,keystone:INSERT=5 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:15.450942 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=48 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:16.333219 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251e2dad-e628-46a7-8055-81bf385ffdee service cinder] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:16.394166 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251e2dad-e628-46a7-8055-81bf385ffdee service cinder] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:16.398675 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9630c9a-97c2-44fc-b62f-29586bded4a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:16.399294 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9630c9a-97c2-44fc-b62f-29586bded4a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:16.399570 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9630c9a-97c2-44fc-b62f-29586bded4a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:16.399847 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9630c9a-97c2-44fc-b62f-29586bded4a9 admin admin] PATH_INFO: `/v3/users/f9dc549329e740feb28d4f04caa2958f` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:16.400953 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9630c9a-97c2-44fc-b62f-29586bded4a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9dc549329e740feb28d4f04caa2958f)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:16.402904 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9630c9a-97c2-44fc-b62f-29586bded4a9 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:16.447996 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-e9630c9a-97c2-44fc-b62f-29586bded4a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9dc549329e740feb28d4f04caa2958f'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:16.460612 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-e9630c9a-97c2-44fc-b62f-29586bded4a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9dc549329e740feb28d4f04caa2958f'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:16.467507 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-e9630c9a-97c2-44fc-b62f-29586bded4a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9dc549329e740feb28d4f04caa2958f'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:16.475224 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-e9630c9a-97c2-44fc-b62f-29586bded4a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9dc549329e740feb28d4f04caa2958f'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:16.475839 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-e9630c9a-97c2-44fc-b62f-29586bded4a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9dc549329e740feb28d4f04caa2958f'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:16.481649 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1100/2184] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:15:16 2022] DELETE /identity/v3/users/f9dc549329e740feb28d4f04caa2958f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:16.486381 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95835e70-0bca-4bf0-bda2-5b6e4d19da38 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:16.486891 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95835e70-0bca-4bf0-bda2-5b6e4d19da38 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:16.487003 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95835e70-0bca-4bf0-bda2-5b6e4d19da38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:16.505542 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-95835e70-0bca-4bf0-bda2-5b6e4d19da38 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:16.506745 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-95835e70-0bca-4bf0-bda2-5b6e4d19da38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:16.564213 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1085/2185] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:16.573715 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9630c9a-97c2-44fc-b62f-29586bded4a9 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:16.575409 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-e9630c9a-97c2-44fc-b62f-29586bded4a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:16.631133 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9630c9a-97c2-44fc-b62f-29586bded4a9 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:16.634776 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b102e74-5523-4805-bf4b-1ccff50b306b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:16.635374 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b102e74-5523-4805-bf4b-1ccff50b306b service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:16.635652 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b102e74-5523-4805-bf4b-1ccff50b306b service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:16.635922 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b102e74-5523-4805-bf4b-1ccff50b306b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:16.645169 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b102e74-5523-4805-bf4b-1ccff50b306b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:16.646614 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b102e74-5523-4805-bf4b-1ccff50b306b service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:16.655786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1101/2186] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:15:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:15:16.912046 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95835e70-0bca-4bf0-bda2-5b6e4d19da38 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:16.913594 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-95835e70-0bca-4bf0-bda2-5b6e4d19da38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:16.925939 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95835e70-0bca-4bf0-bda2-5b6e4d19da38 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:16.930744 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071a147-5577-4ce7-9b5c-1f1ec6cd8cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:16.931436 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071a147-5577-4ce7-9b5c-1f1ec6cd8cf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:16.931784 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071a147-5577-4ce7-9b5c-1f1ec6cd8cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:16.932130 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3071a147-5577-4ce7-9b5c-1f1ec6cd8cf6 admin admin] PATH_INFO: `/v3/projects/38c70ab7ca2044fca7d84782193ae8a3` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:16.933321 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3071a147-5577-4ce7-9b5c-1f1ec6cd8cf6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38c70ab7ca2044fca7d84782193ae8a3)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:16.935406 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3071a147-5577-4ce7-9b5c-1f1ec6cd8cf6 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:16.983490 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-3071a147-5577-4ce7-9b5c-1f1ec6cd8cf6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38c70ab7ca2044fca7d84782193ae8a3', 'request_id': 'req-3071a147-5577-4ce7-9b5c-1f1ec6cd8cf6'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:16.989911 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-3071a147-5577-4ce7-9b5c-1f1ec6cd8cf6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38c70ab7ca2044fca7d84782193ae8a3', 'request_id': 'req-3071a147-5577-4ce7-9b5c-1f1ec6cd8cf6'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:16.995608 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-3071a147-5577-4ce7-9b5c-1f1ec6cd8cf6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38c70ab7ca2044fca7d84782193ae8a3', 'request_id': 'req-3071a147-5577-4ce7-9b5c-1f1ec6cd8cf6'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:17.007146 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-3071a147-5577-4ce7-9b5c-1f1ec6cd8cf6 admin admin] The token cache is being invalidate because project 38c70ab7ca2044fca7d84782193ae8a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:15:17.007431 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-3071a147-5577-4ce7-9b5c-1f1ec6cd8cf6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:17.008052 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-3071a147-5577-4ce7-9b5c-1f1ec6cd8cf6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:17.009533 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1086/2187] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:15:16 2022] DELETE /identity/v3/projects/38c70ab7ca2044fca7d84782193ae8a3 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:17.013496 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b102e74-5523-4805-bf4b-1ccff50b306b service neutron] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:17.024119 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b102e74-5523-4805-bf4b-1ccff50b306b service neutron] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:17.042181 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:17.090937 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37864a0-2db7-4922-b542-a2870616b827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:17.091489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37864a0-2db7-4922-b542-a2870616b827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:17.091767 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37864a0-2db7-4922-b542-a2870616b827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:17.092038 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37864a0-2db7-4922-b542-a2870616b827 admin admin] PATH_INFO: `/v3/users/f57f9751c5954ef4ab103ae51dd9a7f2` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:17.093149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37864a0-2db7-4922-b542-a2870616b827 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f57f9751c5954ef4ab103ae51dd9a7f2)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:17.095149 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37864a0-2db7-4922-b542-a2870616b827 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:17.140934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c37864a0-2db7-4922-b542-a2870616b827 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f57f9751c5954ef4ab103ae51dd9a7f2'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:17.153614 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c37864a0-2db7-4922-b542-a2870616b827 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f57f9751c5954ef4ab103ae51dd9a7f2'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:17.160702 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c37864a0-2db7-4922-b542-a2870616b827 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f57f9751c5954ef4ab103ae51dd9a7f2'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:17.167381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c37864a0-2db7-4922-b542-a2870616b827 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f57f9751c5954ef4ab103ae51dd9a7f2'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:17.168040 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c37864a0-2db7-4922-b542-a2870616b827 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f57f9751c5954ef4ab103ae51dd9a7f2'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:17.174432 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1102/2188] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:15:17 2022] DELETE /identity/v3/users/f57f9751c5954ef4ab103ae51dd9a7f2 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:17.438006 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071a147-5577-4ce7-9b5c-1f1ec6cd8cf6 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:17.439468 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3071a147-5577-4ce7-9b5c-1f1ec6cd8cf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:17.452461 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3071a147-5577-4ce7-9b5c-1f1ec6cd8cf6 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:17.508663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fedf419-503d-483a-9101-21fe4171130f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:17.509295 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedf419-503d-483a-9101-21fe4171130f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:17.509546 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedf419-503d-483a-9101-21fe4171130f admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:17.509813 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fedf419-503d-483a-9101-21fe4171130f admin admin] PATH_INFO: `/v3/projects/7fffc43ffc2d404aa022a40fbfcdbc78` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:17.510922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fedf419-503d-483a-9101-21fe4171130f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fffc43ffc2d404aa022a40fbfcdbc78)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:17.512798 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fedf419-503d-483a-9101-21fe4171130f admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:17.541845 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:17.560085 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-6fedf419-503d-483a-9101-21fe4171130f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fffc43ffc2d404aa022a40fbfcdbc78', 'request_id': 'req-6fedf419-503d-483a-9101-21fe4171130f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:17.566637 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-6fedf419-503d-483a-9101-21fe4171130f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fffc43ffc2d404aa022a40fbfcdbc78', 'request_id': 'req-6fedf419-503d-483a-9101-21fe4171130f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:17.572480 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-6fedf419-503d-483a-9101-21fe4171130f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fffc43ffc2d404aa022a40fbfcdbc78', 'request_id': 'req-6fedf419-503d-483a-9101-21fe4171130f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:17.583741 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-6fedf419-503d-483a-9101-21fe4171130f admin admin] The token cache is being invalidate because project 7fffc43ffc2d404aa022a40fbfcdbc78 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:15:17.584046 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-6fedf419-503d-483a-9101-21fe4171130f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:17.584664 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-6fedf419-503d-483a-9101-21fe4171130f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:17.586133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1087/2189] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:15:17 2022] DELETE /identity/v3/projects/7fffc43ffc2d404aa022a40fbfcdbc78 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:17.590973 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b09aa3-2962-4567-857f-f4ab2f4be7c0 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:17.591434 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b09aa3-2962-4567-857f-f4ab2f4be7c0 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:17.591547 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b09aa3-2962-4567-857f-f4ab2f4be7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:17.611684 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-39b09aa3-2962-4567-857f-f4ab2f4be7c0 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:17.612860 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-39b09aa3-2962-4567-857f-f4ab2f4be7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:17.669185 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1103/2190] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:15:17.677507 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ec612-19a9-4163-aafa-cc9ba3f1e930 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:17.677791 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ec612-19a9-4163-aafa-cc9ba3f1e930 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:17.678064 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ec612-19a9-4163-aafa-cc9ba3f1e930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:17.697254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-4a5ec612-19a9-4163-aafa-cc9ba3f1e930 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:17.698459 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4a5ec612-19a9-4163-aafa-cc9ba3f1e930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:17.706425 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1088/2191] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:17.710509 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b09aa3-2962-4567-857f-f4ab2f4be7c0 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:17.720776 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b09aa3-2962-4567-857f-f4ab2f4be7c0 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:17.725370 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c108c-3a62-4037-9cf4-4a8399dea4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:17.725970 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c108c-3a62-4037-9cf4-4a8399dea4bb admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:17.726268 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c108c-3a62-4037-9cf4-4a8399dea4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:17.726558 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472c108c-3a62-4037-9cf4-4a8399dea4bb admin admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:17.727203 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c108c-3a62-4037-9cf4-4a8399dea4bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:15:17.727507 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c108c-3a62-4037-9cf4-4a8399dea4bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:17.729095 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472c108c-3a62-4037-9cf4-4a8399dea4bb admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:17.741042 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1104/2192] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:15:17.745165 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ec612-19a9-4163-aafa-cc9ba3f1e930 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:17.754210 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ec612-19a9-4163-aafa-cc9ba3f1e930 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:17.758783 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc76d93d-cc70-48f8-a904-f0ab395fb035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:17.759364 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc76d93d-cc70-48f8-a904-f0ab395fb035 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:17.759639 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc76d93d-cc70-48f8-a904-f0ab395fb035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:17.759911 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc76d93d-cc70-48f8-a904-f0ab395fb035 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:17.760536 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc76d93d-cc70-48f8-a904-f0ab395fb035 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:17.762444 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc76d93d-cc70-48f8-a904-f0ab395fb035 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:17.803846 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1089/2193] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:15:17 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:15:17.804956 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c108c-3a62-4037-9cf4-4a8399dea4bb admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:17.814872 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472c108c-3a62-4037-9cf4-4a8399dea4bb admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:17.840792 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:17.873297 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2268a81-b7b4-4ba9-9b36-c54764e69222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:17.873906 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2268a81-b7b4-4ba9-9b36-c54764e69222 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:17.874189 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2268a81-b7b4-4ba9-9b36-c54764e69222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:17.874497 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2268a81-b7b4-4ba9-9b36-c54764e69222 admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:17.875154 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2268a81-b7b4-4ba9-9b36-c54764e69222 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:17.877082 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2268a81-b7b4-4ba9-9b36-c54764e69222 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:17.898707 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1105/2194] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:15:17 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:15:17.902677 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc76d93d-cc70-48f8-a904-f0ab395fb035 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:17.913245 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc76d93d-cc70-48f8-a904-f0ab395fb035 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:17.917859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8b109-9772-4551-a421-192c975fc1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:17.918561 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8b109-9772-4551-a421-192c975fc1d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:17.918844 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8b109-9772-4551-a421-192c975fc1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:17.919115 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c8b109-9772-4551-a421-192c975fc1d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:17.919809 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c8b109-9772-4551-a421-192c975fc1d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:17.921363 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c8b109-9772-4551-a421-192c975fc1d9 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:17.931511 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1090/2195] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:15:17.935461 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2268a81-b7b4-4ba9-9b36-c54764e69222 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:17.945323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2268a81-b7b4-4ba9-9b36-c54764e69222 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:17.950136 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33db1e32-4b03-4b0a-a85c-03f73d228ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:17.950764 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db1e32-4b03-4b0a-a85c-03f73d228ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:17.951040 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db1e32-4b03-4b0a-a85c-03f73d228ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:17.951311 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33db1e32-4b03-4b0a-a85c-03f73d228ad5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:17.951971 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db1e32-4b03-4b0a-a85c-03f73d228ad5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:17.954050 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33db1e32-4b03-4b0a-a85c-03f73d228ad5 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:17.964547 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1106/2196] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:15:17.968407 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8b109-9772-4551-a421-192c975fc1d9 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:17.978633 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c8b109-9772-4551-a421-192c975fc1d9 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:17.983132 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9614b88-2506-481d-b169-1a0de89bf49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:17.983770 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9614b88-2506-481d-b169-1a0de89bf49a admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:17.984047 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9614b88-2506-481d-b169-1a0de89bf49a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:17.984319 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9614b88-2506-481d-b169-1a0de89bf49a admin admin] PATH_INFO: `/v3/projects/40b359a578f04f00b4ebe9bc2c1b6b11/users/db7c49808b0d41bca3b5676313f2e9b4/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:17.998585 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9614b88-2506-481d-b169-1a0de89bf49a admin admin] RBAC: Authorizing `identity:create_grant(project_id=40b359a578f04f00b4ebe9bc2c1b6b11, user_id=db7c49808b0d41bca3b5676313f2e9b4, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:18.001270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9614b88-2506-481d-b169-1a0de89bf49a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:18.013867 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1091/2197] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:15:17 2022] PUT /identity/v3/projects/40b359a578f04f00b4ebe9bc2c1b6b11/users/db7c49808b0d41bca3b5676313f2e9b4/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:18.024915 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d348cc-6942-4190-abca-308e97d28e79 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:18.025205 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d348cc-6942-4190-abca-308e97d28e79 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:18.025484 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d348cc-6942-4190-abca-308e97d28e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:18.053543 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-18d348cc-6942-4190-abca-308e97d28e79 None None] MFA Rules not processed for user `db7c49808b0d41bca3b5676313f2e9b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:18.054812 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-18d348cc-6942-4190-abca-308e97d28e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:18.118139 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1107/2198] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:18.123862 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9614b88-2506-481d-b169-1a0de89bf49a admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:18.186516 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9614b88-2506-481d-b169-1a0de89bf49a admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:18.191580 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f95a45-9127-49c4-bc3f-7f038e3b338b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:18.192184 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f95a45-9127-49c4-bc3f-7f038e3b338b admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:18.192464 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f95a45-9127-49c4-bc3f-7f038e3b338b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:18.192736 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f95a45-9127-49c4-bc3f-7f038e3b338b admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:18.193361 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f95a45-9127-49c4-bc3f-7f038e3b338b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:18.195299 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f95a45-9127-49c4-bc3f-7f038e3b338b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:18.221993 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1092/2199] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:15:18 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:15:18.226129 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d348cc-6942-4190-abca-308e97d28e79 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:18.236917 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d348cc-6942-4190-abca-308e97d28e79 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:18.292466 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705f240-7571-41a4-aedb-e87fc7deb41c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:18.293068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705f240-7571-41a4-aedb-e87fc7deb41c admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:18.293349 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705f240-7571-41a4-aedb-e87fc7deb41c admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:18.293621 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d705f240-7571-41a4-aedb-e87fc7deb41c admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:18.294281 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705f240-7571-41a4-aedb-e87fc7deb41c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:18.296188 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d705f240-7571-41a4-aedb-e87fc7deb41c admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:18.317869 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1108/2200] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:15:18 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:15:18.321917 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f95a45-9127-49c4-bc3f-7f038e3b338b admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:18.333085 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f95a45-9127-49c4-bc3f-7f038e3b338b admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:18.337568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e63df6-f4c3-4bfb-9185-18aa2a7adade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:18.338159 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e63df6-f4c3-4bfb-9185-18aa2a7adade admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:18.338466 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e63df6-f4c3-4bfb-9185-18aa2a7adade admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:18.338738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e63df6-f4c3-4bfb-9185-18aa2a7adade admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:18.339373 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e63df6-f4c3-4bfb-9185-18aa2a7adade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:18.340951 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e63df6-f4c3-4bfb-9185-18aa2a7adade admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:18.351317 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1093/2201] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:15:18.355235 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705f240-7571-41a4-aedb-e87fc7deb41c admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:18.364994 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d705f240-7571-41a4-aedb-e87fc7deb41c admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:18.369691 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc09eb61-4702-40ca-ad42-a7cbd9b2eb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:18.370510 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc09eb61-4702-40ca-ad42-a7cbd9b2eb75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:18.370790 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc09eb61-4702-40ca-ad42-a7cbd9b2eb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:18.371058 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc09eb61-4702-40ca-ad42-a7cbd9b2eb75 admin admin] PATH_INFO: `/v3/projects/ac40caf336624235bb8b8418d5447746/users/03de4f40502f49c6a1e0de185924879a/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:18.386737 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc09eb61-4702-40ca-ad42-a7cbd9b2eb75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac40caf336624235bb8b8418d5447746, user_id=03de4f40502f49c6a1e0de185924879a, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:18.389499 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc09eb61-4702-40ca-ad42-a7cbd9b2eb75 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:18.402538 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1109/2202] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:15:18 2022] PUT /identity/v3/projects/ac40caf336624235bb8b8418d5447746/users/03de4f40502f49c6a1e0de185924879a/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:15:18.406359 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e63df6-f4c3-4bfb-9185-18aa2a7adade admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:18.416534 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e63df6-f4c3-4bfb-9185-18aa2a7adade admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:18.428487 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:18.474525 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca1f23-2d93-467f-a88c-662020ceeb9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:18.475084 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca1f23-2d93-467f-a88c-662020ceeb9a admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:18.475361 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca1f23-2d93-467f-a88c-662020ceeb9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:18.475663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca1f23-2d93-467f-a88c-662020ceeb9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:18.476295 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca1f23-2d93-467f-a88c-662020ceeb9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:18.477867 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcca1f23-2d93-467f-a88c-662020ceeb9a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:18.487482 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1094/2203] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:15:18.491673 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc09eb61-4702-40ca-ad42-a7cbd9b2eb75 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:18.496987 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:18.503747 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc09eb61-4702-40ca-ad42-a7cbd9b2eb75 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:18.508827 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23caef-a13d-4755-8adb-8c37fa6a9333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:18.509505 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23caef-a13d-4755-8adb-8c37fa6a9333 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:18.509782 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23caef-a13d-4755-8adb-8c37fa6a9333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:18.510052 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b23caef-a13d-4755-8adb-8c37fa6a9333 admin admin] PATH_INFO: `/v3/projects/ac40caf336624235bb8b8418d5447746/users/03de4f40502f49c6a1e0de185924879a/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:18.512023 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b23caef-a13d-4755-8adb-8c37fa6a9333 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac40caf336624235bb8b8418d5447746, user_id=03de4f40502f49c6a1e0de185924879a, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:18.514752 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b23caef-a13d-4755-8adb-8c37fa6a9333 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:18.525329 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1110/2204] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:15:18 2022] PUT /identity/v3/projects/ac40caf336624235bb8b8418d5447746/users/03de4f40502f49c6a1e0de185924879a/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:18.536300 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00f25c-aa8e-4301-96b2-16ca866e13bd None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:18.536799 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00f25c-aa8e-4301-96b2-16ca866e13bd None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:18.536914 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a00f25c-aa8e-4301-96b2-16ca866e13bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:18.565326 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-1a00f25c-aa8e-4301-96b2-16ca866e13bd None None] MFA Rules not processed for user `03de4f40502f49c6a1e0de185924879a`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:18.566586 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-1a00f25c-aa8e-4301-96b2-16ca866e13bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:18.633199 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1095/2205] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:18 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:18.644142 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23caef-a13d-4755-8adb-8c37fa6a9333 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:18.645832 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-4b23caef-a13d-4755-8adb-8c37fa6a9333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:18.702621 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b23caef-a13d-4755-8adb-8c37fa6a9333 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:18.706197 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753e4510-f149-43b4-a70f-f4b2b6ee130b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:18.706823 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e4510-f149-43b4-a70f-f4b2b6ee130b service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:18.707101 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e4510-f149-43b4-a70f-f4b2b6ee130b service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:18.707373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e4510-f149-43b4-a70f-f4b2b6ee130b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:18.716223 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753e4510-f149-43b4-a70f-f4b2b6ee130b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:18.717672 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753e4510-f149-43b4-a70f-f4b2b6ee130b service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:18.727263 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1111/2206] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:15:18 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:15:18.756846 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00f25c-aa8e-4301-96b2-16ca866e13bd None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:18.817637 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a00f25c-aa8e-4301-96b2-16ca866e13bd None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:18.822190 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d7bf57-c1e5-43d3-acf0-d3ebf884c6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:18.822842 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7bf57-c1e5-43d3-acf0-d3ebf884c6ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:18.823119 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7bf57-c1e5-43d3-acf0-d3ebf884c6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:18.823389 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7bf57-c1e5-43d3-acf0-d3ebf884c6ec admin admin] PATH_INFO: `/v3/users/db7c49808b0d41bca3b5676313f2e9b4` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:18.824507 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d7bf57-c1e5-43d3-acf0-d3ebf884c6ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=db7c49808b0d41bca3b5676313f2e9b4)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:18.826717 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d7bf57-c1e5-43d3-acf0-d3ebf884c6ec admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:18.867739 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:18.874838 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-30d7bf57-c1e5-43d3-acf0-d3ebf884c6ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db7c49808b0d41bca3b5676313f2e9b4'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:18.887240 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-30d7bf57-c1e5-43d3-acf0-d3ebf884c6ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db7c49808b0d41bca3b5676313f2e9b4'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:18.894624 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-30d7bf57-c1e5-43d3-acf0-d3ebf884c6ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db7c49808b0d41bca3b5676313f2e9b4'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:18.895238 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-30d7bf57-c1e5-43d3-acf0-d3ebf884c6ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db7c49808b0d41bca3b5676313f2e9b4'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:18.902101 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-30d7bf57-c1e5-43d3-acf0-d3ebf884c6ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db7c49808b0d41bca3b5676313f2e9b4'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:18.908469 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1096/2207] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:15:18 2022] DELETE /identity/v3/users/db7c49808b0d41bca3b5676313f2e9b4 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:18.913080 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbf080-f5fb-4b2d-85c1-2c5b4d305539 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:18.913375 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbf080-f5fb-4b2d-85c1-2c5b4d305539 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:18.913671 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbf080-f5fb-4b2d-85c1-2c5b4d305539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:18.934625 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-92dbf080-f5fb-4b2d-85c1-2c5b4d305539 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:18.935822 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-92dbf080-f5fb-4b2d-85c1-2c5b4d305539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:18.993607 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1112/2208] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:19.003227 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d7bf57-c1e5-43d3-acf0-d3ebf884c6ec admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:19.004631 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-30d7bf57-c1e5-43d3-acf0-d3ebf884c6ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:19.061028 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d7bf57-c1e5-43d3-acf0-d3ebf884c6ec admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:19.064748 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3aab4b-7973-4686-a208-efbd2c424e96 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:19.065348 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3aab4b-7973-4686-a208-efbd2c424e96 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:19.065626 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3aab4b-7973-4686-a208-efbd2c424e96 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:19.065898 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3aab4b-7973-4686-a208-efbd2c424e96 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:19.074662 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3aab4b-7973-4686-a208-efbd2c424e96 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:19.076322 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3aab4b-7973-4686-a208-efbd2c424e96 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:19.085585 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1097/2209] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:15:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:15:19.346884 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dbf080-f5fb-4b2d-85c1-2c5b4d305539 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:19.348336 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-92dbf080-f5fb-4b2d-85c1-2c5b4d305539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:19.362042 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dbf080-f5fb-4b2d-85c1-2c5b4d305539 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:19.366663 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a494977-a9d0-4f24-a790-0d5135ac6b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:19.367291 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a494977-a9d0-4f24-a790-0d5135ac6b6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:19.367578 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a494977-a9d0-4f24-a790-0d5135ac6b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:19.367824 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a494977-a9d0-4f24-a790-0d5135ac6b6a admin admin] PATH_INFO: `/v3/projects/40b359a578f04f00b4ebe9bc2c1b6b11` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:19.369090 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a494977-a9d0-4f24-a790-0d5135ac6b6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=40b359a578f04f00b4ebe9bc2c1b6b11)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:19.370975 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a494977-a9d0-4f24-a790-0d5135ac6b6a admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:19.418741 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-5a494977-a9d0-4f24-a790-0d5135ac6b6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40b359a578f04f00b4ebe9bc2c1b6b11', 'request_id': 'req-5a494977-a9d0-4f24-a790-0d5135ac6b6a'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:19.426798 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-5a494977-a9d0-4f24-a790-0d5135ac6b6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40b359a578f04f00b4ebe9bc2c1b6b11', 'request_id': 'req-5a494977-a9d0-4f24-a790-0d5135ac6b6a'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:19.432824 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-5a494977-a9d0-4f24-a790-0d5135ac6b6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40b359a578f04f00b4ebe9bc2c1b6b11', 'request_id': 'req-5a494977-a9d0-4f24-a790-0d5135ac6b6a'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:19.444028 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-5a494977-a9d0-4f24-a790-0d5135ac6b6a admin admin] The token cache is being invalidate because project 40b359a578f04f00b4ebe9bc2c1b6b11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:15:19.444311 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-5a494977-a9d0-4f24-a790-0d5135ac6b6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:19.445045 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-5a494977-a9d0-4f24-a790-0d5135ac6b6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:19.446657 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1113/2210] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:15:19 2022] DELETE /identity/v3/projects/40b359a578f04f00b4ebe9bc2c1b6b11 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:19.450909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3aab4b-7973-4686-a208-efbd2c424e96 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:19.460830 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3aab4b-7973-4686-a208-efbd2c424e96 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:19.515607 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b9806c-9a68-4429-9ead-0cfc3c20a21a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:19.516674 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9806c-9a68-4429-9ead-0cfc3c20a21a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:19.516954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9806c-9a68-4429-9ead-0cfc3c20a21a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:19.517227 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9806c-9a68-4429-9ead-0cfc3c20a21a admin admin] PATH_INFO: `/v3/users/03de4f40502f49c6a1e0de185924879a` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:19.518377 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b9806c-9a68-4429-9ead-0cfc3c20a21a admin admin] RBAC: Authorizing `identity:delete_user(user_id=03de4f40502f49c6a1e0de185924879a)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:19.520314 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b9806c-9a68-4429-9ead-0cfc3c20a21a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:19.565604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-49b9806c-9a68-4429-9ead-0cfc3c20a21a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03de4f40502f49c6a1e0de185924879a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:19.575802 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:19.578172 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-49b9806c-9a68-4429-9ead-0cfc3c20a21a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03de4f40502f49c6a1e0de185924879a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:19.586209 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-49b9806c-9a68-4429-9ead-0cfc3c20a21a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03de4f40502f49c6a1e0de185924879a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:19.587161 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-49b9806c-9a68-4429-9ead-0cfc3c20a21a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03de4f40502f49c6a1e0de185924879a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:19.593984 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-49b9806c-9a68-4429-9ead-0cfc3c20a21a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03de4f40502f49c6a1e0de185924879a'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:19.599655 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1098/2211] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:15:19 2022] DELETE /identity/v3/users/03de4f40502f49c6a1e0de185924879a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:15:19.860819 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a494977-a9d0-4f24-a790-0d5135ac6b6a admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:19.862710 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-5a494977-a9d0-4f24-a790-0d5135ac6b6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:19.884908 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a494977-a9d0-4f24-a790-0d5135ac6b6a admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:19.898321 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:19.944456 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de3cbc-540c-4ea4-80ae-7ae8bee77edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:19.945067 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de3cbc-540c-4ea4-80ae-7ae8bee77edf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:19.945346 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de3cbc-540c-4ea4-80ae-7ae8bee77edf admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:19.945646 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de3cbc-540c-4ea4-80ae-7ae8bee77edf admin admin] PATH_INFO: `/v3/projects/ac40caf336624235bb8b8418d5447746` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:19.946788 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3de3cbc-540c-4ea4-80ae-7ae8bee77edf admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac40caf336624235bb8b8418d5447746)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:19.948694 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3de3cbc-540c-4ea4-80ae-7ae8bee77edf admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:19.994387 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-e3de3cbc-540c-4ea4-80ae-7ae8bee77edf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac40caf336624235bb8b8418d5447746', 'request_id': 'req-e3de3cbc-540c-4ea4-80ae-7ae8bee77edf'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:20.001205 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-e3de3cbc-540c-4ea4-80ae-7ae8bee77edf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac40caf336624235bb8b8418d5447746', 'request_id': 'req-e3de3cbc-540c-4ea4-80ae-7ae8bee77edf'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:20.006825 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-e3de3cbc-540c-4ea4-80ae-7ae8bee77edf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac40caf336624235bb8b8418d5447746', 'request_id': 'req-e3de3cbc-540c-4ea4-80ae-7ae8bee77edf'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:20.018765 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-e3de3cbc-540c-4ea4-80ae-7ae8bee77edf admin admin] The token cache is being invalidate because project ac40caf336624235bb8b8418d5447746 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:15:20.019057 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-e3de3cbc-540c-4ea4-80ae-7ae8bee77edf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:20.019706 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-e3de3cbc-540c-4ea4-80ae-7ae8bee77edf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:20.021531 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1114/2212] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:15:19 2022] DELETE /identity/v3/projects/ac40caf336624235bb8b8418d5447746 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:15:20.026409 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86253555-2cd8-43df-8775-8ba1c8697890 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:20.026701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86253555-2cd8-43df-8775-8ba1c8697890 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:20.026980 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86253555-2cd8-43df-8775-8ba1c8697890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:20.047255 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-86253555-2cd8-43df-8775-8ba1c8697890 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:20.048388 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-86253555-2cd8-43df-8775-8ba1c8697890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:20.106719 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1099/2213] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:20.114482 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb991ac7-b8e7-4cc0-922c-d2c924541c19 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:20.114767 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb991ac7-b8e7-4cc0-922c-d2c924541c19 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:20.115037 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb991ac7-b8e7-4cc0-922c-d2c924541c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:20.134405 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-fb991ac7-b8e7-4cc0-922c-d2c924541c19 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:20.135594 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-fb991ac7-b8e7-4cc0-922c-d2c924541c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:20.144082 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1115/2214] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:20.148122 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86253555-2cd8-43df-8775-8ba1c8697890 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:20.158631 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86253555-2cd8-43df-8775-8ba1c8697890 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:20.163249 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df540607-a15c-412c-a3f9-8dcda882d1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:20.163848 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df540607-a15c-412c-a3f9-8dcda882d1c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:20.164130 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df540607-a15c-412c-a3f9-8dcda882d1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:20.164406 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df540607-a15c-412c-a3f9-8dcda882d1c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:20.165054 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df540607-a15c-412c-a3f9-8dcda882d1c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:15:20.165357 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df540607-a15c-412c-a3f9-8dcda882d1c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:20.166995 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df540607-a15c-412c-a3f9-8dcda882d1c4 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:20.181108 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1100/2215] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:15:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:15:20.185741 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb991ac7-b8e7-4cc0-922c-d2c924541c19 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:20.194983 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb991ac7-b8e7-4cc0-922c-d2c924541c19 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:20.199626 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4fe4c2-03ac-4546-bf53-4750d0baf0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:20.200495 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4fe4c2-03ac-4546-bf53-4750d0baf0ef admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:20.200764 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4fe4c2-03ac-4546-bf53-4750d0baf0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:20.201033 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4fe4c2-03ac-4546-bf53-4750d0baf0ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:20.201658 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4fe4c2-03ac-4546-bf53-4750d0baf0ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:20.204098 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4fe4c2-03ac-4546-bf53-4750d0baf0ef admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:20.231294 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1116/2216] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:15:20 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:15:20.235415 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df540607-a15c-412c-a3f9-8dcda882d1c4 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:20.245241 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df540607-a15c-412c-a3f9-8dcda882d1c4 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:20.300888 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c2215b-c84f-4ee5-ae41-9e28c18c4672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:20.301413 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2215b-c84f-4ee5-ae41-9e28c18c4672 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:20.301719 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2215b-c84f-4ee5-ae41-9e28c18c4672 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:20.301993 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c2215b-c84f-4ee5-ae41-9e28c18c4672 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:20.302660 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c2215b-c84f-4ee5-ae41-9e28c18c4672 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:20.304587 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c2215b-c84f-4ee5-ae41-9e28c18c4672 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:20.325271 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1101/2217] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:15:20 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:15:20.329933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4fe4c2-03ac-4546-bf53-4750d0baf0ef admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:20.339986 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4fe4c2-03ac-4546-bf53-4750d0baf0ef admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:20.345047 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cfa50b-dcf0-45e4-93c6-7c02e57cadb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:20.345640 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cfa50b-dcf0-45e4-93c6-7c02e57cadb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:20.345912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cfa50b-dcf0-45e4-93c6-7c02e57cadb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:20.346176 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cfa50b-dcf0-45e4-93c6-7c02e57cadb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:20.346845 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cfa50b-dcf0-45e4-93c6-7c02e57cadb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:20.348408 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cfa50b-dcf0-45e4-93c6-7c02e57cadb0 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:20.358073 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1117/2218] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 20:15:20.361822 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c2215b-c84f-4ee5-ae41-9e28c18c4672 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:20.372057 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c2215b-c84f-4ee5-ae41-9e28c18c4672 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:20.376648 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602d5388-6e62-47be-8562-2bed75524f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:20.377267 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d5388-6e62-47be-8562-2bed75524f9f admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:20.377547 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d5388-6e62-47be-8562-2bed75524f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:20.377821 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602d5388-6e62-47be-8562-2bed75524f9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:20.378496 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d5388-6e62-47be-8562-2bed75524f9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:20.380430 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602d5388-6e62-47be-8562-2bed75524f9f admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:20.391485 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1102/2219] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:15:20.395416 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cfa50b-dcf0-45e4-93c6-7c02e57cadb0 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:20.405262 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cfa50b-dcf0-45e4-93c6-7c02e57cadb0 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:20.410403 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b5604-6a77-4382-be34-27b35461a8b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:20.411058 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b5604-6a77-4382-be34-27b35461a8b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:20.411332 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b5604-6a77-4382-be34-27b35461a8b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:20.411600 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b5604-6a77-4382-be34-27b35461a8b3 admin admin] PATH_INFO: `/v3/projects/75259472c54145bd8d7c873bd5917775/users/3cc819a8150b4f6c8abe6d4872b92e73/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:20.425855 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b5604-6a77-4382-be34-27b35461a8b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75259472c54145bd8d7c873bd5917775, user_id=3cc819a8150b4f6c8abe6d4872b92e73, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:20.428528 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b5604-6a77-4382-be34-27b35461a8b3 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:20.441408 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1118/2220] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:15:20 2022] PUT /identity/v3/projects/75259472c54145bd8d7c873bd5917775/users/3cc819a8150b4f6c8abe6d4872b92e73/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:20.451889 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00962999-8e6d-4384-888d-5c7826117ded None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:20.452181 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00962999-8e6d-4384-888d-5c7826117ded None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:20.452457 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00962999-8e6d-4384-888d-5c7826117ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:20.480282 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-00962999-8e6d-4384-888d-5c7826117ded None None] MFA Rules not processed for user `3cc819a8150b4f6c8abe6d4872b92e73`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:20.481499 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-00962999-8e6d-4384-888d-5c7826117ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:20.513825 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:20.547065 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1103/2221] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:20.558226 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b5604-6a77-4382-be34-27b35461a8b3 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:20.559928 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-711b5604-6a77-4382-be34-27b35461a8b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:20.605176 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:20.617449 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b5604-6a77-4382-be34-27b35461a8b3 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:20.621021 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30acf694-0547-49c4-a2d7-bc288bfb35ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:20.621615 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30acf694-0547-49c4-a2d7-bc288bfb35ec service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:20.621892 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30acf694-0547-49c4-a2d7-bc288bfb35ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:20.622164 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30acf694-0547-49c4-a2d7-bc288bfb35ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:20.630942 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30acf694-0547-49c4-a2d7-bc288bfb35ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:20.632351 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30acf694-0547-49c4-a2d7-bc288bfb35ec service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:20.640840 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1119/2222] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:15:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:15:20.660644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00962999-8e6d-4384-888d-5c7826117ded None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:20.722793 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00962999-8e6d-4384-888d-5c7826117ded None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:20.727398 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac9f2c-272c-431f-aa40-dc7e8b34a74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:20.728022 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac9f2c-272c-431f-aa40-dc7e8b34a74b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:20.728299 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac9f2c-272c-431f-aa40-dc7e8b34a74b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:20.728570 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac9f2c-272c-431f-aa40-dc7e8b34a74b admin admin] PATH_INFO: `/v3/users/3cc819a8150b4f6c8abe6d4872b92e73` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:20.729679 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac9f2c-272c-431f-aa40-dc7e8b34a74b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cc819a8150b4f6c8abe6d4872b92e73)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:20.731629 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac9f2c-272c-431f-aa40-dc7e8b34a74b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:20.776897 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d8ac9f2c-272c-431f-aa40-dc7e8b34a74b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cc819a8150b4f6c8abe6d4872b92e73'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:20.788652 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d8ac9f2c-272c-431f-aa40-dc7e8b34a74b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cc819a8150b4f6c8abe6d4872b92e73'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:20.795500 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d8ac9f2c-272c-431f-aa40-dc7e8b34a74b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cc819a8150b4f6c8abe6d4872b92e73'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:20.796136 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d8ac9f2c-272c-431f-aa40-dc7e8b34a74b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cc819a8150b4f6c8abe6d4872b92e73'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:20.803580 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d8ac9f2c-272c-431f-aa40-dc7e8b34a74b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cc819a8150b4f6c8abe6d4872b92e73'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:20.809239 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1104/2223] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:15:20 2022] DELETE /identity/v3/users/3cc819a8150b4f6c8abe6d4872b92e73 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:20.814418 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19071dc6-7a52-44a7-b90f-3bfd7ba93b0c None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:20.814706 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19071dc6-7a52-44a7-b90f-3bfd7ba93b0c None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:20.814985 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19071dc6-7a52-44a7-b90f-3bfd7ba93b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:20.833882 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-19071dc6-7a52-44a7-b90f-3bfd7ba93b0c None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:20.835101 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-19071dc6-7a52-44a7-b90f-3bfd7ba93b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:20.892478 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1120/2224] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:20.902062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac9f2c-272c-431f-aa40-dc7e8b34a74b admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:20.903519 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d8ac9f2c-272c-431f-aa40-dc7e8b34a74b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:20.961263 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac9f2c-272c-431f-aa40-dc7e8b34a74b admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:20.964843 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7af981-add0-4cd5-bdaa-36a89c2d79f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:20.965439 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7af981-add0-4cd5-bdaa-36a89c2d79f4 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:20.965718 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7af981-add0-4cd5-bdaa-36a89c2d79f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:20.965990 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7af981-add0-4cd5-bdaa-36a89c2d79f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:20.972732 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:20.975557 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7af981-add0-4cd5-bdaa-36a89c2d79f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:20.977333 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7af981-add0-4cd5-bdaa-36a89c2d79f4 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:20.989085 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1105/2225] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:15:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:15:21.250652 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19071dc6-7a52-44a7-b90f-3bfd7ba93b0c None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:21.252070 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-19071dc6-7a52-44a7-b90f-3bfd7ba93b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:21.263942 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19071dc6-7a52-44a7-b90f-3bfd7ba93b0c None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:21.268953 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5ebeb-aae6-4d46-ba61-897a7335a2ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:21.269661 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5ebeb-aae6-4d46-ba61-897a7335a2ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:21.270007 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5ebeb-aae6-4d46-ba61-897a7335a2ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:21.270391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5ebeb-aae6-4d46-ba61-897a7335a2ca admin admin] PATH_INFO: `/v3/projects/75259472c54145bd8d7c873bd5917775` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:21.271589 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5ebeb-aae6-4d46-ba61-897a7335a2ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=75259472c54145bd8d7c873bd5917775)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:21.273495 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5ebeb-aae6-4d46-ba61-897a7335a2ca admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:21.320068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-37e5ebeb-aae6-4d46-ba61-897a7335a2ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75259472c54145bd8d7c873bd5917775', 'request_id': 'req-37e5ebeb-aae6-4d46-ba61-897a7335a2ca'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:21.326597 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-37e5ebeb-aae6-4d46-ba61-897a7335a2ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75259472c54145bd8d7c873bd5917775', 'request_id': 'req-37e5ebeb-aae6-4d46-ba61-897a7335a2ca'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:21.332377 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-37e5ebeb-aae6-4d46-ba61-897a7335a2ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75259472c54145bd8d7c873bd5917775', 'request_id': 'req-37e5ebeb-aae6-4d46-ba61-897a7335a2ca'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:21.343268 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-37e5ebeb-aae6-4d46-ba61-897a7335a2ca admin admin] The token cache is being invalidate because project 75259472c54145bd8d7c873bd5917775 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:15:21.343565 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-37e5ebeb-aae6-4d46-ba61-897a7335a2ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:21.344703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-37e5ebeb-aae6-4d46-ba61-897a7335a2ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:21.346181 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1121/2226] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:15:21 2022] DELETE /identity/v3/projects/75259472c54145bd8d7c873bd5917775 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:21.351077 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced33700-1473-49c4-ae32-5cc759b8d222 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:21.351366 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced33700-1473-49c4-ae32-5cc759b8d222 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:21.351661 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced33700-1473-49c4-ae32-5cc759b8d222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:21.371707 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-ced33700-1473-49c4-ae32-5cc759b8d222 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:21.372903 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ced33700-1473-49c4-ae32-5cc759b8d222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:21.430397 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1106/2227] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:21.439150 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5300e07-f7ec-46c7-a56f-05464f3bfdf1 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:21.439443 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5300e07-f7ec-46c7-a56f-05464f3bfdf1 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:21.439722 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5300e07-f7ec-46c7-a56f-05464f3bfdf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:21.460414 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-b5300e07-f7ec-46c7-a56f-05464f3bfdf1 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:21.461619 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-b5300e07-f7ec-46c7-a56f-05464f3bfdf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:21.470060 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1122/2228] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:15:21.474177 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced33700-1473-49c4-ae32-5cc759b8d222 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:21.483923 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced33700-1473-49c4-ae32-5cc759b8d222 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:21.489139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ddfb3f-2c3f-437a-b04c-0addf84b4d65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:21.489731 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ddfb3f-2c3f-437a-b04c-0addf84b4d65 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:21.490007 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ddfb3f-2c3f-437a-b04c-0addf84b4d65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:21.490306 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ddfb3f-2c3f-437a-b04c-0addf84b4d65 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:21.490977 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ddfb3f-2c3f-437a-b04c-0addf84b4d65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:15:21.491274 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ddfb3f-2c3f-437a-b04c-0addf84b4d65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:21.492848 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ddfb3f-2c3f-437a-b04c-0addf84b4d65 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:21.504143 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1107/2229] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:15:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:15:21.508217 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5300e07-f7ec-46c7-a56f-05464f3bfdf1 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:21.517857 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5300e07-f7ec-46c7-a56f-05464f3bfdf1 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:21.522499 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf75e331-5293-4659-b56a-963c3908ab0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:21.523088 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf75e331-5293-4659-b56a-963c3908ab0a admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:21.523365 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf75e331-5293-4659-b56a-963c3908ab0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:21.523634 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf75e331-5293-4659-b56a-963c3908ab0a admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:21.524298 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf75e331-5293-4659-b56a-963c3908ab0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:21.526169 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf75e331-5293-4659-b56a-963c3908ab0a admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:21.552483 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1123/2230] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:15:21 2022] POST /identity/v3/projects => generated 361 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:15:21.556702 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ddfb3f-2c3f-437a-b04c-0addf84b4d65 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:21.566460 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ddfb3f-2c3f-437a-b04c-0addf84b4d65 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:21.621778 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3c1fc-54a7-4caf-a291-d4d2317cade5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:21.622320 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3c1fc-54a7-4caf-a291-d4d2317cade5 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:21.622599 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3c1fc-54a7-4caf-a291-d4d2317cade5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:21.622868 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3c1fc-54a7-4caf-a291-d4d2317cade5 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:21.623491 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d3c1fc-54a7-4caf-a291-d4d2317cade5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:21.625396 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d3c1fc-54a7-4caf-a291-d4d2317cade5 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:21.646194 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1108/2231] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:15:21 2022] POST /identity/v3/users => generated 281 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:15:21.650220 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf75e331-5293-4659-b56a-963c3908ab0a admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:21.661106 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf75e331-5293-4659-b56a-963c3908ab0a admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:21.665735 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b71ef-be86-41f8-b31d-9f1cd51ebf8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:21.666361 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b71ef-be86-41f8-b31d-9f1cd51ebf8a admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:21.666639 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b71ef-be86-41f8-b31d-9f1cd51ebf8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:21.666907 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30b71ef-be86-41f8-b31d-9f1cd51ebf8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:21.667546 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b71ef-be86-41f8-b31d-9f1cd51ebf8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:21.669114 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30b71ef-be86-41f8-b31d-9f1cd51ebf8a admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:21.679378 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1124/2232] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:15:21.683080 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3c1fc-54a7-4caf-a291-d4d2317cade5 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:21.692781 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3c1fc-54a7-4caf-a291-d4d2317cade5 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:21.697377 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189acd21-c5ef-4cfa-a24d-307ccbf9a952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:21.697959 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189acd21-c5ef-4cfa-a24d-307ccbf9a952 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:21.698235 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189acd21-c5ef-4cfa-a24d-307ccbf9a952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:21.698530 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189acd21-c5ef-4cfa-a24d-307ccbf9a952 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:21.699164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189acd21-c5ef-4cfa-a24d-307ccbf9a952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:21.700988 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189acd21-c5ef-4cfa-a24d-307ccbf9a952 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:21.711923 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1109/2233] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:15:21.715704 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b71ef-be86-41f8-b31d-9f1cd51ebf8a admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:21.726129 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30b71ef-be86-41f8-b31d-9f1cd51ebf8a admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:21.730742 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe87e62-49e0-4f02-b2a9-8c8afd1d61d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:21.731380 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe87e62-49e0-4f02-b2a9-8c8afd1d61d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:21.731660 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe87e62-49e0-4f02-b2a9-8c8afd1d61d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:21.731932 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe87e62-49e0-4f02-b2a9-8c8afd1d61d0 admin admin] PATH_INFO: `/v3/projects/8dc834c34b1f499ca8882c54d70cb71e/users/2a435062f6b94af597e03f03833e8dc7/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:21.746154 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe87e62-49e0-4f02-b2a9-8c8afd1d61d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dc834c34b1f499ca8882c54d70cb71e, user_id=2a435062f6b94af597e03f03833e8dc7, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:21.748829 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe87e62-49e0-4f02-b2a9-8c8afd1d61d0 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:21.761416 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1125/2234] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:15:21 2022] PUT /identity/v3/projects/8dc834c34b1f499ca8882c54d70cb71e/users/2a435062f6b94af597e03f03833e8dc7/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:21.771919 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e18349-abe8-4fa2-b935-e30edc8e2656 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:21.772199 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e18349-abe8-4fa2-b935-e30edc8e2656 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:21.772468 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e18349-abe8-4fa2-b935-e30edc8e2656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:21.800002 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-48e18349-abe8-4fa2-b935-e30edc8e2656 None None] MFA Rules not processed for user `2a435062f6b94af597e03f03833e8dc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:21.801198 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-48e18349-abe8-4fa2-b935-e30edc8e2656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:21.848156 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:21.865193 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1110/2235] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:21.899145 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe87e62-49e0-4f02-b2a9-8c8afd1d61d0 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:21.900951 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-6fe87e62-49e0-4f02-b2a9-8c8afd1d61d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:21.908367 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:21.961541 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe87e62-49e0-4f02-b2a9-8c8afd1d61d0 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:21.965180 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a72f563-1a31-4bcb-bad4-4fe3d0034446 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:21.965915 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72f563-1a31-4bcb-bad4-4fe3d0034446 service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:21.966202 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72f563-1a31-4bcb-bad4-4fe3d0034446 service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:21.966518 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a72f563-1a31-4bcb-bad4-4fe3d0034446 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:21.974885 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a72f563-1a31-4bcb-bad4-4fe3d0034446 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:21.976356 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a72f563-1a31-4bcb-bad4-4fe3d0034446 service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:21.985240 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1126/2236] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:15:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:15:22.001526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e18349-abe8-4fa2-b935-e30edc8e2656 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:22.063996 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e18349-abe8-4fa2-b935-e30edc8e2656 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:22.068478 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a934a0-e906-41e4-b51e-8cd1c4d14b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:22.069100 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a934a0-e906-41e4-b51e-8cd1c4d14b3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:22.069382 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a934a0-e906-41e4-b51e-8cd1c4d14b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:22.069655 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a934a0-e906-41e4-b51e-8cd1c4d14b3d admin admin] PATH_INFO: `/v3/users/2a435062f6b94af597e03f03833e8dc7` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:22.070803 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a934a0-e906-41e4-b51e-8cd1c4d14b3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a435062f6b94af597e03f03833e8dc7)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:22.072709 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a934a0-e906-41e4-b51e-8cd1c4d14b3d admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:22.117216 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-55a934a0-e906-41e4-b51e-8cd1c4d14b3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a435062f6b94af597e03f03833e8dc7'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:22.129560 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-55a934a0-e906-41e4-b51e-8cd1c4d14b3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a435062f6b94af597e03f03833e8dc7'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:22.136699 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-55a934a0-e906-41e4-b51e-8cd1c4d14b3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a435062f6b94af597e03f03833e8dc7'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:22.137343 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-55a934a0-e906-41e4-b51e-8cd1c4d14b3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a435062f6b94af597e03f03833e8dc7'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:22.145035 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-55a934a0-e906-41e4-b51e-8cd1c4d14b3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a435062f6b94af597e03f03833e8dc7'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:22.150840 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1111/2237] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:15:21 2022] DELETE /identity/v3/users/2a435062f6b94af597e03f03833e8dc7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:22.155587 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bac24a-fe81-4e0d-9e1e-faee4c334cd5 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:22.155883 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bac24a-fe81-4e0d-9e1e-faee4c334cd5 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:22.156161 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bac24a-fe81-4e0d-9e1e-faee4c334cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:22.175594 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-41bac24a-fe81-4e0d-9e1e-faee4c334cd5 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:22.177127 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-41bac24a-fe81-4e0d-9e1e-faee4c334cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:22.236858 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1127/2238] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:22.246534 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a934a0-e906-41e4-b51e-8cd1c4d14b3d admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:22.247915 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-55a934a0-e906-41e4-b51e-8cd1c4d14b3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:22.305168 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a934a0-e906-41e4-b51e-8cd1c4d14b3d admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:22.309327 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3050c2c9-ee57-490d-a2ae-895b009f1a74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:22.309924 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050c2c9-ee57-490d-a2ae-895b009f1a74 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:22.310205 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050c2c9-ee57-490d-a2ae-895b009f1a74 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:22.310506 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3050c2c9-ee57-490d-a2ae-895b009f1a74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:22.319660 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3050c2c9-ee57-490d-a2ae-895b009f1a74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:22.321076 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3050c2c9-ee57-490d-a2ae-895b009f1a74 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:22.331153 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1112/2239] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:15:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:15:22.588091 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bac24a-fe81-4e0d-9e1e-faee4c334cd5 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:22.589528 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-41bac24a-fe81-4e0d-9e1e-faee4c334cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:22.601318 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bac24a-fe81-4e0d-9e1e-faee4c334cd5 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:22.605884 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f24fde0-ab41-446f-83c6-ba2bcaee0618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:22.606546 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24fde0-ab41-446f-83c6-ba2bcaee0618 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:22.606823 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24fde0-ab41-446f-83c6-ba2bcaee0618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:22.607093 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f24fde0-ab41-446f-83c6-ba2bcaee0618 admin admin] PATH_INFO: `/v3/projects/8dc834c34b1f499ca8882c54d70cb71e` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:22.608213 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f24fde0-ab41-446f-83c6-ba2bcaee0618 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dc834c34b1f499ca8882c54d70cb71e)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:22.610695 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f24fde0-ab41-446f-83c6-ba2bcaee0618 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:22.657522 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-1f24fde0-ab41-446f-83c6-ba2bcaee0618 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dc834c34b1f499ca8882c54d70cb71e', 'request_id': 'req-1f24fde0-ab41-446f-83c6-ba2bcaee0618'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:22.664824 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-1f24fde0-ab41-446f-83c6-ba2bcaee0618 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dc834c34b1f499ca8882c54d70cb71e', 'request_id': 'req-1f24fde0-ab41-446f-83c6-ba2bcaee0618'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:22.670451 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-1f24fde0-ab41-446f-83c6-ba2bcaee0618 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dc834c34b1f499ca8882c54d70cb71e', 'request_id': 'req-1f24fde0-ab41-446f-83c6-ba2bcaee0618'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:22.682080 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-1f24fde0-ab41-446f-83c6-ba2bcaee0618 admin admin] The token cache is being invalidate because project 8dc834c34b1f499ca8882c54d70cb71e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:15:22.682402 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-1f24fde0-ab41-446f-83c6-ba2bcaee0618 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:22.683028 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-1f24fde0-ab41-446f-83c6-ba2bcaee0618 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:22.684498 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1128/2240] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:15:22 2022] DELETE /identity/v3/projects/8dc834c34b1f499ca8882c54d70cb71e => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:22.689297 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421dad5e-bf43-47dd-9e24-2271bc5065a1 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:22.689774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421dad5e-bf43-47dd-9e24-2271bc5065a1 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:22.689886 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421dad5e-bf43-47dd-9e24-2271bc5065a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:22.709255 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-421dad5e-bf43-47dd-9e24-2271bc5065a1 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:22.710492 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-421dad5e-bf43-47dd-9e24-2271bc5065a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:22.718840 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:22.771553 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1113/2241] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:22.779163 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ba339-181d-44f3-9870-43562e88013a None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:22.779449 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ba339-181d-44f3-9870-43562e88013a None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:22.779754 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94ba339-181d-44f3-9870-43562e88013a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:22.800096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-c94ba339-181d-44f3-9870-43562e88013a None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:22.801294 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-c94ba339-181d-44f3-9870-43562e88013a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:22.809131 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1129/2242] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:22.813172 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421dad5e-bf43-47dd-9e24-2271bc5065a1 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:22.822704 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421dad5e-bf43-47dd-9e24-2271bc5065a1 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:22.827526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b779231a-2ff2-4826-9405-d2e034cf30ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:22.828119 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779231a-2ff2-4826-9405-d2e034cf30ca admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:22.828400 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779231a-2ff2-4826-9405-d2e034cf30ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:22.828707 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b779231a-2ff2-4826-9405-d2e034cf30ca admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:22.829352 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b779231a-2ff2-4826-9405-d2e034cf30ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:15:22.829664 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b779231a-2ff2-4826-9405-d2e034cf30ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:22.831290 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b779231a-2ff2-4826-9405-d2e034cf30ca admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:22.842825 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1114/2243] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:15:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:15:22.847356 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94ba339-181d-44f3-9870-43562e88013a None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:22.856335 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94ba339-181d-44f3-9870-43562e88013a None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:22.861472 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67faf89e-a285-460c-82d3-2bd2551d4ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:22.862060 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67faf89e-a285-460c-82d3-2bd2551d4ef0 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:22.862374 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67faf89e-a285-460c-82d3-2bd2551d4ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:22.862647 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67faf89e-a285-460c-82d3-2bd2551d4ef0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:22.863340 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67faf89e-a285-460c-82d3-2bd2551d4ef0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:22.865217 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67faf89e-a285-460c-82d3-2bd2551d4ef0 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:22.893681 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1130/2244] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:15:22 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:15:22.897760 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b779231a-2ff2-4826-9405-d2e034cf30ca admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:22.908343 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b779231a-2ff2-4826-9405-d2e034cf30ca admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:22.966830 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd70ce-1fc7-445e-9095-20a538bbb9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:22.967566 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd70ce-1fc7-445e-9095-20a538bbb9f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:22.967866 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd70ce-1fc7-445e-9095-20a538bbb9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:22.968142 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd70ce-1fc7-445e-9095-20a538bbb9f9 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:22.968784 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd70ce-1fc7-445e-9095-20a538bbb9f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:22.970728 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd70ce-1fc7-445e-9095-20a538bbb9f9 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:22.993955 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1115/2245] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:15:22 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:15:22.995130 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67faf89e-a285-460c-82d3-2bd2551d4ef0 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:23.001282 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:23.007859 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67faf89e-a285-460c-82d3-2bd2551d4ef0 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:23.013476 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6248d1-68fa-4777-926c-de170aa3c334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:23.014192 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6248d1-68fa-4777-926c-de170aa3c334 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:23.014517 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6248d1-68fa-4777-926c-de170aa3c334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:23.014794 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6248d1-68fa-4777-926c-de170aa3c334 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:23.015452 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6248d1-68fa-4777-926c-de170aa3c334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:23.017624 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6248d1-68fa-4777-926c-de170aa3c334 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:23.027935 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1131/2246] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:22 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 20:15:23.032009 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd70ce-1fc7-445e-9095-20a538bbb9f9 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:23.043072 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd70ce-1fc7-445e-9095-20a538bbb9f9 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:23.049755 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f6980e-ed42-4ced-9800-ce6b281d6c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:23.050443 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6980e-ed42-4ced-9800-ce6b281d6c4d admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:23.050754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6980e-ed42-4ced-9800-ce6b281d6c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:23.051025 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f6980e-ed42-4ced-9800-ce6b281d6c4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:23.051676 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f6980e-ed42-4ced-9800-ce6b281d6c4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:23.053272 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f6980e-ed42-4ced-9800-ce6b281d6c4d admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:23.064900 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1116/2247] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:15:23.069066 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6248d1-68fa-4777-926c-de170aa3c334 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:23.079399 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6248d1-68fa-4777-926c-de170aa3c334 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:23.084051 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30b77d-6d91-4d76-bcec-b97c0764321c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:23.084699 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30b77d-6d91-4d76-bcec-b97c0764321c admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:23.084980 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30b77d-6d91-4d76-bcec-b97c0764321c admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:23.085253 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30b77d-6d91-4d76-bcec-b97c0764321c admin admin] PATH_INFO: `/v3/projects/6f33ce26965d45beb8990a8c6affac40/users/353f35382f23462ea44d595475dd5a60/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:23.099607 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e30b77d-6d91-4d76-bcec-b97c0764321c admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f33ce26965d45beb8990a8c6affac40, user_id=353f35382f23462ea44d595475dd5a60, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:23.102291 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e30b77d-6d91-4d76-bcec-b97c0764321c admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:23.115055 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1132/2248] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:15:23 2022] PUT /identity/v3/projects/6f33ce26965d45beb8990a8c6affac40/users/353f35382f23462ea44d595475dd5a60/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:23.179806 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2c9d2-3a61-48ab-850d-1328ffc621e9 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:23.180005 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2c9d2-3a61-48ab-850d-1328ffc621e9 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:23.180294 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c2c9d2-3a61-48ab-850d-1328ffc621e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:23.208830 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-21c2c9d2-3a61-48ab-850d-1328ffc621e9 None None] MFA Rules not processed for user `353f35382f23462ea44d595475dd5a60`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:23.210043 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-21c2c9d2-3a61-48ab-850d-1328ffc621e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:23.272491 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1117/2249] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:23.283989 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30b77d-6d91-4d76-bcec-b97c0764321c admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:23.285784 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-8e30b77d-6d91-4d76-bcec-b97c0764321c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:23.342130 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30b77d-6d91-4d76-bcec-b97c0764321c admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:23.345784 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9982c-c738-48d9-b8fa-5804cd490858 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '021216f90d7f4ff6b6eb7a1d86399d99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:23.346421 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9982c-c738-48d9-b8fa-5804cd490858 service glance] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:23.346705 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9982c-c738-48d9-b8fa-5804cd490858 service glance] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:23.346979 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b9982c-c738-48d9-b8fa-5804cd490858 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:23.355330 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b9982c-c738-48d9-b8fa-5804cd490858 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:23.356750 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b9982c-c738-48d9-b8fa-5804cd490858 service glance] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:23.365977 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1133/2250] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:15:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 20:15:30.121790 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2c9d2-3a61-48ab-850d-1328ffc621e9 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:30.123181 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-21c2c9d2-3a61-48ab-850d-1328ffc621e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:30.133488 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:30.179489 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c2c9d2-3a61-48ab-850d-1328ffc621e9 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:30.183012 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe9fa1-446b-4411-85a8-dbe84eddfe02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:30.183607 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe9fa1-446b-4411-85a8-dbe84eddfe02 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:30.183881 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe9fa1-446b-4411-85a8-dbe84eddfe02 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:30.184150 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe9fa1-446b-4411-85a8-dbe84eddfe02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:30.185821 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-65fe9fa1-446b-4411-85a8-dbe84eddfe02 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:30.273615 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fe9fa1-446b-4411-85a8-dbe84eddfe02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:30.274981 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fe9fa1-446b-4411-85a8-dbe84eddfe02 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:30.293251 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1118/2251] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:15:30 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:15:30.641862 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9982c-c738-48d9-b8fa-5804cd490858 service glance] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:30.643881 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-29b9982c-c738-48d9-b8fa-5804cd490858 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:30.699443 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b9982c-c738-48d9-b8fa-5804cd490858 service glance] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:30.703339 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89b634-d217-47af-8a62-2cf8b35fbe45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:30.703938 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89b634-d217-47af-8a62-2cf8b35fbe45 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:30.704221 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89b634-d217-47af-8a62-2cf8b35fbe45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:30.704493 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c89b634-d217-47af-8a62-2cf8b35fbe45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:30.706828 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-5c89b634-d217-47af-8a62-2cf8b35fbe45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:30.782083 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c89b634-d217-47af-8a62-2cf8b35fbe45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:30.783542 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c89b634-d217-47af-8a62-2cf8b35fbe45 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:30.794235 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1134/2252] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:15:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:15:31.991048 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe9fa1-446b-4411-85a8-dbe84eddfe02 service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:32.001176 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe9fa1-446b-4411-85a8-dbe84eddfe02 service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:32.005416 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490c4f4f-5bb7-4744-8b3b-332dd1fc5086 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:32.006022 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c4f4f-5bb7-4744-8b3b-332dd1fc5086 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:32.006333 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c4f4f-5bb7-4744-8b3b-332dd1fc5086 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:32.006610 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490c4f4f-5bb7-4744-8b3b-332dd1fc5086 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:32.008350 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-490c4f4f-5bb7-4744-8b3b-332dd1fc5086 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:32.024015 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490c4f4f-5bb7-4744-8b3b-332dd1fc5086 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:32.025462 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490c4f4f-5bb7-4744-8b3b-332dd1fc5086 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:32.038052 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1119/2253] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:15:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:15:37.387200 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89b634-d217-47af-8a62-2cf8b35fbe45 service neutron] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:37.395724 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c89b634-d217-47af-8a62-2cf8b35fbe45 service neutron] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:37.399400 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb5c7a1-f438-47a1-8a77-6c74d4b09b84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:37.399993 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5c7a1-f438-47a1-8a77-6c74d4b09b84 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:37.400275 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5c7a1-f438-47a1-8a77-6c74d4b09b84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:37.400545 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb5c7a1-f438-47a1-8a77-6c74d4b09b84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:37.402223 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-fcb5c7a1-f438-47a1-8a77-6c74d4b09b84 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:37.411703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:37.468530 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb5c7a1-f438-47a1-8a77-6c74d4b09b84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:37.469963 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb5c7a1-f438-47a1-8a77-6c74d4b09b84 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:37.480654 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1135/2254] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:15:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:15:40.605772 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490c4f4f-5bb7-4744-8b3b-332dd1fc5086 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:40.607435 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-490c4f4f-5bb7-4744-8b3b-332dd1fc5086 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:40.628282 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490c4f4f-5bb7-4744-8b3b-332dd1fc5086 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:40.632414 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9198fc1-84ca-4f69-b232-f8c0421309cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:40.633158 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9198fc1-84ca-4f69-b232-f8c0421309cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:40.633505 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9198fc1-84ca-4f69-b232-f8c0421309cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:40.633932 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9198fc1-84ca-4f69-b232-f8c0421309cf admin admin] PATH_INFO: `/v3/users/7cef68ee160348dea427e6c46b775e2d` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:40.635106 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9198fc1-84ca-4f69-b232-f8c0421309cf admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cef68ee160348dea427e6c46b775e2d)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:40.637030 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9198fc1-84ca-4f69-b232-f8c0421309cf admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:40.689029 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e9198fc1-84ca-4f69-b232-f8c0421309cf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cef68ee160348dea427e6c46b775e2d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:40.701126 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e9198fc1-84ca-4f69-b232-f8c0421309cf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cef68ee160348dea427e6c46b775e2d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:40.708011 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e9198fc1-84ca-4f69-b232-f8c0421309cf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cef68ee160348dea427e6c46b775e2d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:40.708610 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e9198fc1-84ca-4f69-b232-f8c0421309cf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cef68ee160348dea427e6c46b775e2d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:40.713155 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:40.716975 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e9198fc1-84ca-4f69-b232-f8c0421309cf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cef68ee160348dea427e6c46b775e2d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:40.724456 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1120/2255] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:15:40 2022] DELETE /identity/v3/users/7cef68ee160348dea427e6c46b775e2d => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:40.893615 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb5c7a1-f438-47a1-8a77-6c74d4b09b84 service neutron] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:40.905366 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb5c7a1-f438-47a1-8a77-6c74d4b09b84 service neutron] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:40.963641 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d20e11c-a9fd-47e3-a301-7476ee1d4827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:40.964267 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20e11c-a9fd-47e3-a301-7476ee1d4827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:40.964543 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20e11c-a9fd-47e3-a301-7476ee1d4827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:40.964813 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20e11c-a9fd-47e3-a301-7476ee1d4827 admin admin] PATH_INFO: `/v3/projects/4023fb7431ad496fa044b19190f014d8` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:40.965924 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d20e11c-a9fd-47e3-a301-7476ee1d4827 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4023fb7431ad496fa044b19190f014d8)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:40.967962 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d20e11c-a9fd-47e3-a301-7476ee1d4827 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:41.013345 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-6d20e11c-a9fd-47e3-a301-7476ee1d4827 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4023fb7431ad496fa044b19190f014d8', 'request_id': 'req-6d20e11c-a9fd-47e3-a301-7476ee1d4827'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:41.020842 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-6d20e11c-a9fd-47e3-a301-7476ee1d4827 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4023fb7431ad496fa044b19190f014d8', 'request_id': 'req-6d20e11c-a9fd-47e3-a301-7476ee1d4827'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:41.026455 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-6d20e11c-a9fd-47e3-a301-7476ee1d4827 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4023fb7431ad496fa044b19190f014d8', 'request_id': 'req-6d20e11c-a9fd-47e3-a301-7476ee1d4827'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:41.037977 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-6d20e11c-a9fd-47e3-a301-7476ee1d4827 admin admin] The token cache is being invalidate because project 4023fb7431ad496fa044b19190f014d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:15:41.038353 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-6d20e11c-a9fd-47e3-a301-7476ee1d4827 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:41.039011 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-6d20e11c-a9fd-47e3-a301-7476ee1d4827 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:41.040533 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1136/2256] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:15:40 2022] DELETE /identity/v3/projects/4023fb7431ad496fa044b19190f014d8 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:41.045758 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cba178-2dde-4fce-88f6-e6f6736f2c8b None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:41.046035 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cba178-2dde-4fce-88f6-e6f6736f2c8b None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:41.046356 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cba178-2dde-4fce-88f6-e6f6736f2c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:41.066007 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-19cba178-2dde-4fce-88f6-e6f6736f2c8b None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:41.067234 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-19cba178-2dde-4fce-88f6-e6f6736f2c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:41.124388 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1121/2257] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:41.132648 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b18cff-8565-4bcd-95d8-8659a60e9931 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:41.132930 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b18cff-8565-4bcd-95d8-8659a60e9931 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:41.133201 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b18cff-8565-4bcd-95d8-8659a60e9931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:41.153312 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-32b18cff-8565-4bcd-95d8-8659a60e9931 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:41.154537 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-32b18cff-8565-4bcd-95d8-8659a60e9931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:41.162438 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1137/2258] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:41.166607 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cba178-2dde-4fce-88f6-e6f6736f2c8b None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:41.176283 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cba178-2dde-4fce-88f6-e6f6736f2c8b None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:41.180845 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f3b80a-1f8e-486e-8099-f754a3cc3771 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:41.181439 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3b80a-1f8e-486e-8099-f754a3cc3771 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:41.181716 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3b80a-1f8e-486e-8099-f754a3cc3771 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:41.181986 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f3b80a-1f8e-486e-8099-f754a3cc3771 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:41.182660 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f3b80a-1f8e-486e-8099-f754a3cc3771 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:15:41.182962 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f3b80a-1f8e-486e-8099-f754a3cc3771 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:41.184548 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f3b80a-1f8e-486e-8099-f754a3cc3771 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:41.196356 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1122/2259] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:15:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:15:41.200918 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b18cff-8565-4bcd-95d8-8659a60e9931 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:41.210694 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b18cff-8565-4bcd-95d8-8659a60e9931 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:41.215343 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344b1fc-e915-45bb-8fb1-205bad667085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:41.215933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344b1fc-e915-45bb-8fb1-205bad667085 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:41.216211 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344b1fc-e915-45bb-8fb1-205bad667085 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:41.216485 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e344b1fc-e915-45bb-8fb1-205bad667085 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:41.217109 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e344b1fc-e915-45bb-8fb1-205bad667085 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:41.219038 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e344b1fc-e915-45bb-8fb1-205bad667085 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:41.245307 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1138/2260] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:15:41 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:15:41.249507 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f3b80a-1f8e-486e-8099-f754a3cc3771 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:41.259826 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f3b80a-1f8e-486e-8099-f754a3cc3771 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:41.317461 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a0a51-f316-4ef6-8899-60516d373a0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:41.318860 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a0a51-f316-4ef6-8899-60516d373a0a admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:41.319144 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a0a51-f316-4ef6-8899-60516d373a0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:41.319607 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a0a51-f316-4ef6-8899-60516d373a0a admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:41.320267 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88a0a51-f316-4ef6-8899-60516d373a0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:41.322292 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88a0a51-f316-4ef6-8899-60516d373a0a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:41.345028 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1123/2261] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:15:41 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:15:41.348827 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344b1fc-e915-45bb-8fb1-205bad667085 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:41.358944 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e344b1fc-e915-45bb-8fb1-205bad667085 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:41.364091 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b86ca8-404a-48ba-b08b-ee220c4ca723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:41.364690 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b86ca8-404a-48ba-b08b-ee220c4ca723 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:41.364971 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b86ca8-404a-48ba-b08b-ee220c4ca723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:41.365244 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b86ca8-404a-48ba-b08b-ee220c4ca723 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:41.365894 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b86ca8-404a-48ba-b08b-ee220c4ca723 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:41.367525 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b86ca8-404a-48ba-b08b-ee220c4ca723 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:41.368836 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:41.381080 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1139/2262] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:15:41.383754 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a0a51-f316-4ef6-8899-60516d373a0a admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:41.395344 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a0a51-f316-4ef6-8899-60516d373a0a admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:41.399982 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5285ef9-7d1c-4594-9709-c9552058759f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:41.400577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5285ef9-7d1c-4594-9709-c9552058759f admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:41.400985 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5285ef9-7d1c-4594-9709-c9552058759f admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:41.401128 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5285ef9-7d1c-4594-9709-c9552058759f admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:41.401771 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5285ef9-7d1c-4594-9709-c9552058759f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:41.403405 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5285ef9-7d1c-4594-9709-c9552058759f admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:41.414546 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1124/2263] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:15:41.418604 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b86ca8-404a-48ba-b08b-ee220c4ca723 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:41.428577 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b86ca8-404a-48ba-b08b-ee220c4ca723 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:41.433110 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee0784-3e72-49e7-8ce1-8bbfd345b23a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:41.433755 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee0784-3e72-49e7-8ce1-8bbfd345b23a admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:41.434035 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee0784-3e72-49e7-8ce1-8bbfd345b23a admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:41.434341 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ee0784-3e72-49e7-8ce1-8bbfd345b23a admin admin] PATH_INFO: `/v3/projects/16a4f3b3ce6d4b91835c2ce7a031bb1e/users/85000f00f4424141a2c2885abb2bb9cc/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:41.448518 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ee0784-3e72-49e7-8ce1-8bbfd345b23a admin admin] RBAC: Authorizing `identity:create_grant(project_id=16a4f3b3ce6d4b91835c2ce7a031bb1e, user_id=85000f00f4424141a2c2885abb2bb9cc, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:41.451471 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ee0784-3e72-49e7-8ce1-8bbfd345b23a admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:41.464587 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1140/2264] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:15:41 2022] PUT /identity/v3/projects/16a4f3b3ce6d4b91835c2ce7a031bb1e/users/85000f00f4424141a2c2885abb2bb9cc/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:41.476809 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3bd68-c5c5-4d91-9e93-04c7fd08e598 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:41.477098 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3bd68-c5c5-4d91-9e93-04c7fd08e598 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:41.477417 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e3bd68-c5c5-4d91-9e93-04c7fd08e598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:41.505607 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-57e3bd68-c5c5-4d91-9e93-04c7fd08e598 None None] MFA Rules not processed for user `85000f00f4424141a2c2885abb2bb9cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:41.506878 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-57e3bd68-c5c5-4d91-9e93-04c7fd08e598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:41.552653 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:41.570422 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1125/2265] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:41 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:41.574995 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee0784-3e72-49e7-8ce1-8bbfd345b23a admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:41.635589 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ee0784-3e72-49e7-8ce1-8bbfd345b23a admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:41.640187 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb6d9a-784a-4f72-a5bf-a87754cd061b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:41.640791 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb6d9a-784a-4f72-a5bf-a87754cd061b admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:41.641078 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb6d9a-784a-4f72-a5bf-a87754cd061b admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:41.641352 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb6d9a-784a-4f72-a5bf-a87754cd061b admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:41.641990 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb6d9a-784a-4f72-a5bf-a87754cd061b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:41.644510 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb6d9a-784a-4f72-a5bf-a87754cd061b admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:41.670865 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1141/2266] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:15:41 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:15:41.675138 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e3bd68-c5c5-4d91-9e93-04c7fd08e598 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:41.687571 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e3bd68-c5c5-4d91-9e93-04c7fd08e598 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:41.745393 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dbbe3d-fa88-456e-8678-b09f051f05ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:41.745996 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbbe3d-fa88-456e-8678-b09f051f05ce admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:41.746292 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbbe3d-fa88-456e-8678-b09f051f05ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:41.746567 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3dbbe3d-fa88-456e-8678-b09f051f05ce admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:41.747189 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dbbe3d-fa88-456e-8678-b09f051f05ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:41.749089 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3dbbe3d-fa88-456e-8678-b09f051f05ce admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:41.770338 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1126/2267] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:15:41 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:15:41.774511 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb6d9a-784a-4f72-a5bf-a87754cd061b admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:41.785120 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb6d9a-784a-4f72-a5bf-a87754cd061b admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:41.789740 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81bc3e-c920-4e23-b253-be53880cb887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:41.790365 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81bc3e-c920-4e23-b253-be53880cb887 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:41.790646 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81bc3e-c920-4e23-b253-be53880cb887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:41.790918 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a81bc3e-c920-4e23-b253-be53880cb887 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:41.791564 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a81bc3e-c920-4e23-b253-be53880cb887 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:41.793142 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a81bc3e-c920-4e23-b253-be53880cb887 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:41.803593 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1142/2268] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:15:41.807549 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dbbe3d-fa88-456e-8678-b09f051f05ce admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:41.817949 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3dbbe3d-fa88-456e-8678-b09f051f05ce admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:41.822542 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4bfca4-4113-417c-880c-81178c886937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:41.823191 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bfca4-4113-417c-880c-81178c886937 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:41.823467 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bfca4-4113-417c-880c-81178c886937 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:41.823737 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4bfca4-4113-417c-880c-81178c886937 admin admin] PATH_INFO: `/v3/projects/800b5af8128a49f6962df27e2af21c5e/users/24cc0ee528cd43078bd05177b1184095/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:41.839708 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4bfca4-4113-417c-880c-81178c886937 admin admin] RBAC: Authorizing `identity:create_grant(project_id=800b5af8128a49f6962df27e2af21c5e, user_id=24cc0ee528cd43078bd05177b1184095, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:41.842422 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4bfca4-4113-417c-880c-81178c886937 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:41.855804 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1127/2269] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:15:41 2022] PUT /identity/v3/projects/800b5af8128a49f6962df27e2af21c5e/users/24cc0ee528cd43078bd05177b1184095/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:41.859753 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81bc3e-c920-4e23-b253-be53880cb887 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:41.870528 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a81bc3e-c920-4e23-b253-be53880cb887 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:41.926910 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c7f30e-fc88-42b7-aeb9-9192c0cd8a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:41.927508 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7f30e-fc88-42b7-aeb9-9192c0cd8a16 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:41.927697 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7f30e-fc88-42b7-aeb9-9192c0cd8a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:41.927965 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7f30e-fc88-42b7-aeb9-9192c0cd8a16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:41.928601 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c7f30e-fc88-42b7-aeb9-9192c0cd8a16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:41.930184 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c7f30e-fc88-42b7-aeb9-9192c0cd8a16 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:41.939718 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1143/2270] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:41 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:15:41.943817 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4bfca4-4113-417c-880c-81178c886937 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:41.953996 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4bfca4-4113-417c-880c-81178c886937 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:41.958584 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b6dd05-a7c6-43e6-b032-98639c680005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:41.959220 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6dd05-a7c6-43e6-b032-98639c680005 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:41.959492 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6dd05-a7c6-43e6-b032-98639c680005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:41.959760 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6dd05-a7c6-43e6-b032-98639c680005 admin admin] PATH_INFO: `/v3/projects/800b5af8128a49f6962df27e2af21c5e/users/24cc0ee528cd43078bd05177b1184095/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:41.961628 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b6dd05-a7c6-43e6-b032-98639c680005 admin admin] RBAC: Authorizing `identity:create_grant(project_id=800b5af8128a49f6962df27e2af21c5e, user_id=24cc0ee528cd43078bd05177b1184095, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:41.964292 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b6dd05-a7c6-43e6-b032-98639c680005 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:41.974864 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1128/2271] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:15:41 2022] PUT /identity/v3/projects/800b5af8128a49f6962df27e2af21c5e/users/24cc0ee528cd43078bd05177b1184095/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:41.984913 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f5e98-8d52-4bac-8a80-3cd1461317b8 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:41.985206 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f5e98-8d52-4bac-8a80-3cd1461317b8 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:41.985470 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f5e98-8d52-4bac-8a80-3cd1461317b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:42.013473 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-076f5e98-8d52-4bac-8a80-3cd1461317b8 None None] MFA Rules not processed for user `24cc0ee528cd43078bd05177b1184095`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:42.014708 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-076f5e98-8d52-4bac-8a80-3cd1461317b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:42.045182 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:42.085085 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1144/2272] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:41 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:42.095314 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b6dd05-a7c6-43e6-b032-98639c680005 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:42.097524 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-94b6dd05-a7c6-43e6-b032-98639c680005 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:42.153770 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b6dd05-a7c6-43e6-b032-98639c680005 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:42.157577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31daea2-217b-43c8-ac62-28045531063b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:42.158169 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31daea2-217b-43c8-ac62-28045531063b service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:42.158497 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31daea2-217b-43c8-ac62-28045531063b service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:42.159316 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31daea2-217b-43c8-ac62-28045531063b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:42.168108 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31daea2-217b-43c8-ac62-28045531063b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:42.169516 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31daea2-217b-43c8-ac62-28045531063b service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:42.179054 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1129/2273] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:15:42 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:15:42.280442 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f5e98-8d52-4bac-8a80-3cd1461317b8 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:42.290947 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f5e98-8d52-4bac-8a80-3cd1461317b8 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:42.294545 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dd59da-9ba0-41d1-af4b-0ccafb7fa61e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:42.295139 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd59da-9ba0-41d1-af4b-0ccafb7fa61e service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:42.295417 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd59da-9ba0-41d1-af4b-0ccafb7fa61e service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:42.295689 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dd59da-9ba0-41d1-af4b-0ccafb7fa61e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:42.353242 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dd59da-9ba0-41d1-af4b-0ccafb7fa61e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:42.354692 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dd59da-9ba0-41d1-af4b-0ccafb7fa61e service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:42.363375 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1145/2274] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:15:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:15:45.988773 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31daea2-217b-43c8-ac62-28045531063b service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:46.039221 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:46.059236 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31daea2-217b-43c8-ac62-28045531063b service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:46.064481 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ccc5c5-0419-4dc6-9203-1a6b480aac7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:46.065016 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ccc5c5-0419-4dc6-9203-1a6b480aac7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:46.065340 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ccc5c5-0419-4dc6-9203-1a6b480aac7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:46.065713 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ccc5c5-0419-4dc6-9203-1a6b480aac7d admin admin] PATH_INFO: `/v3/users/85000f00f4424141a2c2885abb2bb9cc` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:46.067614 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ccc5c5-0419-4dc6-9203-1a6b480aac7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=85000f00f4424141a2c2885abb2bb9cc)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:46.069743 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ccc5c5-0419-4dc6-9203-1a6b480aac7d admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:46.117525 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-95ccc5c5-0419-4dc6-9203-1a6b480aac7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85000f00f4424141a2c2885abb2bb9cc'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:46.130114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-95ccc5c5-0419-4dc6-9203-1a6b480aac7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85000f00f4424141a2c2885abb2bb9cc'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:46.137604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-95ccc5c5-0419-4dc6-9203-1a6b480aac7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85000f00f4424141a2c2885abb2bb9cc'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:46.138233 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-95ccc5c5-0419-4dc6-9203-1a6b480aac7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85000f00f4424141a2c2885abb2bb9cc'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:46.145440 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-95ccc5c5-0419-4dc6-9203-1a6b480aac7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85000f00f4424141a2c2885abb2bb9cc'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:46.151731 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1130/2275] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:15:45 2022] DELETE /identity/v3/users/85000f00f4424141a2c2885abb2bb9cc => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:46.156134 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a13d1-c970-4946-b8f0-e332b895948f None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:46.156623 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a13d1-c970-4946-b8f0-e332b895948f None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:46.156731 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393a13d1-c970-4946-b8f0-e332b895948f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:46.176418 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-393a13d1-c970-4946-b8f0-e332b895948f None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:46.177656 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-393a13d1-c970-4946-b8f0-e332b895948f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:46.235604 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1146/2276] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:15:46.246111 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ccc5c5-0419-4dc6-9203-1a6b480aac7d admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:46.247549 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-95ccc5c5-0419-4dc6-9203-1a6b480aac7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:46.303805 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ccc5c5-0419-4dc6-9203-1a6b480aac7d admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:46.307447 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474bd6c-88e8-4c69-87fa-eb4af2313210 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:46.308045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474bd6c-88e8-4c69-87fa-eb4af2313210 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:46.308325 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474bd6c-88e8-4c69-87fa-eb4af2313210 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:46.308599 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474bd6c-88e8-4c69-87fa-eb4af2313210 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:46.317373 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474bd6c-88e8-4c69-87fa-eb4af2313210 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:46.318822 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c474bd6c-88e8-4c69-87fa-eb4af2313210 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:46.328363 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1131/2277] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:15:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:15:46.597836 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a13d1-c970-4946-b8f0-e332b895948f None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:46.599347 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-393a13d1-c970-4946-b8f0-e332b895948f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:46.613695 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393a13d1-c970-4946-b8f0-e332b895948f None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:46.617946 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a981a6fd-74bc-42bc-a506-9f44531ed61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:46.619298 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981a6fd-74bc-42bc-a506-9f44531ed61e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:46.619588 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981a6fd-74bc-42bc-a506-9f44531ed61e admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:46.619956 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a981a6fd-74bc-42bc-a506-9f44531ed61e admin admin] PATH_INFO: `/v3/projects/16a4f3b3ce6d4b91835c2ce7a031bb1e` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:46.620963 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a981a6fd-74bc-42bc-a506-9f44531ed61e admin admin] RBAC: Authorizing `identity:delete_project(project_id=16a4f3b3ce6d4b91835c2ce7a031bb1e)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:46.622910 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a981a6fd-74bc-42bc-a506-9f44531ed61e admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:46.669355 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-a981a6fd-74bc-42bc-a506-9f44531ed61e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16a4f3b3ce6d4b91835c2ce7a031bb1e', 'request_id': 'req-a981a6fd-74bc-42bc-a506-9f44531ed61e'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:46.677117 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-a981a6fd-74bc-42bc-a506-9f44531ed61e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16a4f3b3ce6d4b91835c2ce7a031bb1e', 'request_id': 'req-a981a6fd-74bc-42bc-a506-9f44531ed61e'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:46.681026 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:46.683404 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-a981a6fd-74bc-42bc-a506-9f44531ed61e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16a4f3b3ce6d4b91835c2ce7a031bb1e', 'request_id': 'req-a981a6fd-74bc-42bc-a506-9f44531ed61e'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:46.697181 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-a981a6fd-74bc-42bc-a506-9f44531ed61e admin admin] The token cache is being invalidate because project 16a4f3b3ce6d4b91835c2ce7a031bb1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:15:46.697469 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-a981a6fd-74bc-42bc-a506-9f44531ed61e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:46.698100 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-a981a6fd-74bc-42bc-a506-9f44531ed61e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:46.699620 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1147/2278] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:15:46 2022] DELETE /identity/v3/projects/16a4f3b3ce6d4b91835c2ce7a031bb1e => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:46.703843 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474bd6c-88e8-4c69-87fa-eb4af2313210 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:46.713235 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474bd6c-88e8-4c69-87fa-eb4af2313210 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:46.771077 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb58023-0acc-466f-8351-ff36ed64b7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:46.771716 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb58023-0acc-466f-8351-ff36ed64b7e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:46.771995 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb58023-0acc-466f-8351-ff36ed64b7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:46.772271 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb58023-0acc-466f-8351-ff36ed64b7e7 admin admin] PATH_INFO: `/v3/users/24cc0ee528cd43078bd05177b1184095` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:46.773417 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb58023-0acc-466f-8351-ff36ed64b7e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24cc0ee528cd43078bd05177b1184095)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:46.775366 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb58023-0acc-466f-8351-ff36ed64b7e7 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:46.781071 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:46.823515 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-dcb58023-0acc-466f-8351-ff36ed64b7e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24cc0ee528cd43078bd05177b1184095'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:46.835651 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-dcb58023-0acc-466f-8351-ff36ed64b7e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24cc0ee528cd43078bd05177b1184095'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:46.842868 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-dcb58023-0acc-466f-8351-ff36ed64b7e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24cc0ee528cd43078bd05177b1184095'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:46.843488 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-dcb58023-0acc-466f-8351-ff36ed64b7e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24cc0ee528cd43078bd05177b1184095'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:46.850859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-dcb58023-0acc-466f-8351-ff36ed64b7e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24cc0ee528cd43078bd05177b1184095'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:46.856631 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1132/2279] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:15:46 2022] DELETE /identity/v3/users/24cc0ee528cd43078bd05177b1184095 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:47.127574 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a981a6fd-74bc-42bc-a506-9f44531ed61e admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:47.129180 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-a981a6fd-74bc-42bc-a506-9f44531ed61e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:47.150927 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a981a6fd-74bc-42bc-a506-9f44531ed61e admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:47.217403 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80308b6d-25cb-4ba0-9a5d-d3db0a733425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:47.218042 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80308b6d-25cb-4ba0-9a5d-d3db0a733425 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:47.218351 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80308b6d-25cb-4ba0-9a5d-d3db0a733425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:47.218620 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80308b6d-25cb-4ba0-9a5d-d3db0a733425 admin admin] PATH_INFO: `/v3/projects/800b5af8128a49f6962df27e2af21c5e` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:47.219730 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80308b6d-25cb-4ba0-9a5d-d3db0a733425 admin admin] RBAC: Authorizing `identity:delete_project(project_id=800b5af8128a49f6962df27e2af21c5e)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:47.221609 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80308b6d-25cb-4ba0-9a5d-d3db0a733425 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:47.269630 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-80308b6d-25cb-4ba0-9a5d-d3db0a733425 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '800b5af8128a49f6962df27e2af21c5e', 'request_id': 'req-80308b6d-25cb-4ba0-9a5d-d3db0a733425'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:47.277372 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-80308b6d-25cb-4ba0-9a5d-d3db0a733425 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '800b5af8128a49f6962df27e2af21c5e', 'request_id': 'req-80308b6d-25cb-4ba0-9a5d-d3db0a733425'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:47.283169 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-80308b6d-25cb-4ba0-9a5d-d3db0a733425 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '800b5af8128a49f6962df27e2af21c5e', 'request_id': 'req-80308b6d-25cb-4ba0-9a5d-d3db0a733425'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:47.295283 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-80308b6d-25cb-4ba0-9a5d-d3db0a733425 admin admin] The token cache is being invalidate because project 800b5af8128a49f6962df27e2af21c5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:15:47.295595 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-80308b6d-25cb-4ba0-9a5d-d3db0a733425 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:47.296253 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-80308b6d-25cb-4ba0-9a5d-d3db0a733425 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:47.297736 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1148/2280] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:15:47 2022] DELETE /identity/v3/projects/800b5af8128a49f6962df27e2af21c5e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:47.303659 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640bd3e-652c-49df-8238-788d50729d70 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:47.304136 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640bd3e-652c-49df-8238-788d50729d70 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:47.304237 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640bd3e-652c-49df-8238-788d50729d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:47.324589 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-e640bd3e-652c-49df-8238-788d50729d70 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:47.326051 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e640bd3e-652c-49df-8238-788d50729d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:47.391296 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1133/2281] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:47.399031 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f527e-b33a-48d3-b9e1-0ed5f2b80417 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:47.399394 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f527e-b33a-48d3-b9e1-0ed5f2b80417 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:47.399782 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750f527e-b33a-48d3-b9e1-0ed5f2b80417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:47.420732 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-750f527e-b33a-48d3-b9e1-0ed5f2b80417 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:47.422110 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-750f527e-b33a-48d3-b9e1-0ed5f2b80417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:47.431110 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1149/2282] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:47.435624 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640bd3e-652c-49df-8238-788d50729d70 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:47.459343 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640bd3e-652c-49df-8238-788d50729d70 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:47.460106 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ecfb7-6ee8-4688-b616-a3dc395a5641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:47.460713 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ecfb7-6ee8-4688-b616-a3dc395a5641 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:47.460990 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ecfb7-6ee8-4688-b616-a3dc395a5641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:47.461261 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445ecfb7-6ee8-4688-b616-a3dc395a5641 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:47.461905 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445ecfb7-6ee8-4688-b616-a3dc395a5641 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:15:47.462202 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445ecfb7-6ee8-4688-b616-a3dc395a5641 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:47.464442 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445ecfb7-6ee8-4688-b616-a3dc395a5641 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:47.479447 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1134/2283] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:15:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:15:47.481213 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f527e-b33a-48d3-b9e1-0ed5f2b80417 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:47.491322 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750f527e-b33a-48d3-b9e1-0ed5f2b80417 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:47.496312 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324cfd3-d447-4bc2-a384-47e0bd165459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:47.496731 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324cfd3-d447-4bc2-a384-47e0bd165459 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:47.497008 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324cfd3-d447-4bc2-a384-47e0bd165459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:47.497279 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2324cfd3-d447-4bc2-a384-47e0bd165459 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:47.498046 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2324cfd3-d447-4bc2-a384-47e0bd165459 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:47.500009 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2324cfd3-d447-4bc2-a384-47e0bd165459 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:47.542750 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1150/2284] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:15:47 2022] POST /identity/v3/projects => generated 379 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:15:47.546912 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ecfb7-6ee8-4688-b616-a3dc395a5641 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:47.557947 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445ecfb7-6ee8-4688-b616-a3dc395a5641 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:47.614379 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e301dcd-ae14-4d65-a6bb-a8e5972e1d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:47.614977 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e301dcd-ae14-4d65-a6bb-a8e5972e1d43 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:47.615255 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e301dcd-ae14-4d65-a6bb-a8e5972e1d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:47.615524 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e301dcd-ae14-4d65-a6bb-a8e5972e1d43 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:47.616143 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e301dcd-ae14-4d65-a6bb-a8e5972e1d43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:47.618123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e301dcd-ae14-4d65-a6bb-a8e5972e1d43 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:47.638316 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1135/2285] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:15:47 2022] POST /identity/v3/users => generated 290 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:15:47.642453 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324cfd3-d447-4bc2-a384-47e0bd165459 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:47.652955 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2324cfd3-d447-4bc2-a384-47e0bd165459 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:47.657697 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5675b9e-23ca-446d-8557-e70b82c97b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:47.658324 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5675b9e-23ca-446d-8557-e70b82c97b54 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:47.658626 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5675b9e-23ca-446d-8557-e70b82c97b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:47.658899 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5675b9e-23ca-446d-8557-e70b82c97b54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:47.659531 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5675b9e-23ca-446d-8557-e70b82c97b54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:47.661113 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5675b9e-23ca-446d-8557-e70b82c97b54 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:47.670931 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1151/2286] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:15:47.674887 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e301dcd-ae14-4d65-a6bb-a8e5972e1d43 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:47.680529 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:47.686581 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e301dcd-ae14-4d65-a6bb-a8e5972e1d43 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:47.692828 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26de6ce-c152-42a5-9c6c-ddc36cd697b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:47.693426 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26de6ce-c152-42a5-9c6c-ddc36cd697b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:47.693710 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26de6ce-c152-42a5-9c6c-ddc36cd697b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:47.693985 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26de6ce-c152-42a5-9c6c-ddc36cd697b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:47.694687 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26de6ce-c152-42a5-9c6c-ddc36cd697b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:47.696332 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26de6ce-c152-42a5-9c6c-ddc36cd697b6 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:47.707205 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1136/2287] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:15:47.711205 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5675b9e-23ca-446d-8557-e70b82c97b54 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:47.721156 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5675b9e-23ca-446d-8557-e70b82c97b54 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:47.726161 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cc4759-54e6-4e11-a6bf-8d517bf0cd9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:47.726834 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc4759-54e6-4e11-a6bf-8d517bf0cd9e admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:47.727113 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc4759-54e6-4e11-a6bf-8d517bf0cd9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:47.727385 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc4759-54e6-4e11-a6bf-8d517bf0cd9e admin admin] PATH_INFO: `/v3/projects/d6de55c2c5d24a4d9f97763f184e7094/users/269f6ffd85cb446ab4716d7193a0b78f/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:47.742083 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cc4759-54e6-4e11-a6bf-8d517bf0cd9e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6de55c2c5d24a4d9f97763f184e7094, user_id=269f6ffd85cb446ab4716d7193a0b78f, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:47.744770 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cc4759-54e6-4e11-a6bf-8d517bf0cd9e admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:47.757157 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1152/2288] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:15:47 2022] PUT /identity/v3/projects/d6de55c2c5d24a4d9f97763f184e7094/users/269f6ffd85cb446ab4716d7193a0b78f/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:47.768511 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1e228-fa14-444d-b620-60a784f57466 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:47.768803 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1e228-fa14-444d-b620-60a784f57466 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:47.769114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1e228-fa14-444d-b620-60a784f57466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:47.796757 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-98e1e228-fa14-444d-b620-60a784f57466 None None] MFA Rules not processed for user `269f6ffd85cb446ab4716d7193a0b78f`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:47.797970 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-98e1e228-fa14-444d-b620-60a784f57466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:47.861845 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1137/2289] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:47 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:47.866937 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cc4759-54e6-4e11-a6bf-8d517bf0cd9e admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:47.884544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:47.929614 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cc4759-54e6-4e11-a6bf-8d517bf0cd9e admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:47.934356 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee72717-dd13-4859-bad2-5b80ab18fbe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:47.934951 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee72717-dd13-4859-bad2-5b80ab18fbe7 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:47.935228 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee72717-dd13-4859-bad2-5b80ab18fbe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:47.935500 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee72717-dd13-4859-bad2-5b80ab18fbe7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:47.936131 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee72717-dd13-4859-bad2-5b80ab18fbe7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:47.938042 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee72717-dd13-4859-bad2-5b80ab18fbe7 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:47.963673 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1153/2290] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:15:47 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:15:47.967678 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1e228-fa14-444d-b620-60a784f57466 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:47.978823 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1e228-fa14-444d-b620-60a784f57466 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:48.035530 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63950427-d6bb-4453-bf5e-cb3a0f780bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:48.036171 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63950427-d6bb-4453-bf5e-cb3a0f780bee admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:48.036516 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63950427-d6bb-4453-bf5e-cb3a0f780bee admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:48.036791 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63950427-d6bb-4453-bf5e-cb3a0f780bee admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:48.037450 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63950427-d6bb-4453-bf5e-cb3a0f780bee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:48.039407 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63950427-d6bb-4453-bf5e-cb3a0f780bee admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:48.061535 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1138/2291] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:15:47 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:15:48.065542 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee72717-dd13-4859-bad2-5b80ab18fbe7 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:48.075939 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee72717-dd13-4859-bad2-5b80ab18fbe7 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:48.080481 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd977441-ac1b-409c-80e5-a208d3531a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:48.081069 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd977441-ac1b-409c-80e5-a208d3531a09 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:48.081345 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd977441-ac1b-409c-80e5-a208d3531a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:48.081616 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd977441-ac1b-409c-80e5-a208d3531a09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:48.082781 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd977441-ac1b-409c-80e5-a208d3531a09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:48.084362 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd977441-ac1b-409c-80e5-a208d3531a09 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:48.094668 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1154/2292] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:15:48.098879 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63950427-d6bb-4453-bf5e-cb3a0f780bee admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:48.108499 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63950427-d6bb-4453-bf5e-cb3a0f780bee admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:48.113174 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b209860-9f4c-48b9-a1f5-2bb0b4142462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:48.114196 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b209860-9f4c-48b9-a1f5-2bb0b4142462 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:48.114514 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b209860-9f4c-48b9-a1f5-2bb0b4142462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:48.114975 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b209860-9f4c-48b9-a1f5-2bb0b4142462 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:48.115681 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b209860-9f4c-48b9-a1f5-2bb0b4142462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:48.117588 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b209860-9f4c-48b9-a1f5-2bb0b4142462 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:48.127381 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1139/2293] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 20:15:48.131403 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd977441-ac1b-409c-80e5-a208d3531a09 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:48.148062 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd977441-ac1b-409c-80e5-a208d3531a09 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:48.153177 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05cc96d-f854-4b52-8a4d-91feed962189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:48.153846 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05cc96d-f854-4b52-8a4d-91feed962189 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:48.154125 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05cc96d-f854-4b52-8a4d-91feed962189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:48.154457 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05cc96d-f854-4b52-8a4d-91feed962189 admin admin] PATH_INFO: `/v3/projects/c42e7f1939de44dda1b805f4cc73ada6/users/111ebeb244314a04968542a064d6f636/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:48.170164 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05cc96d-f854-4b52-8a4d-91feed962189 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c42e7f1939de44dda1b805f4cc73ada6, user_id=111ebeb244314a04968542a064d6f636, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:48.172856 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05cc96d-f854-4b52-8a4d-91feed962189 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:48.185457 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1155/2294] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:15:48 2022] PUT /identity/v3/projects/c42e7f1939de44dda1b805f4cc73ada6/users/111ebeb244314a04968542a064d6f636/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:48.195756 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b7cb1-2321-4cfc-99ed-ca34d7326a5e None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:48.196046 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b7cb1-2321-4cfc-99ed-ca34d7326a5e None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:48.196324 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b7cb1-2321-4cfc-99ed-ca34d7326a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:48.225355 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-446b7cb1-2321-4cfc-99ed-ca34d7326a5e None None] MFA Rules not processed for user `111ebeb244314a04968542a064d6f636`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:48.226521 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-446b7cb1-2321-4cfc-99ed-ca34d7326a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:48.272573 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:48.291402 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1140/2295] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:48.295889 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05cc96d-f854-4b52-8a4d-91feed962189 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:48.357442 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05cc96d-f854-4b52-8a4d-91feed962189 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:48.361991 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f2316-2542-45fc-bf43-0cd53928a6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:48.362619 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f2316-2542-45fc-bf43-0cd53928a6c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:48.362893 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f2316-2542-45fc-bf43-0cd53928a6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:48.363159 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f2316-2542-45fc-bf43-0cd53928a6c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:48.363807 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3f2316-2542-45fc-bf43-0cd53928a6c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:48.365682 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3f2316-2542-45fc-bf43-0cd53928a6c2 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:48.392775 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1156/2296] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:15:48 2022] POST /identity/v3/projects => generated 379 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:15:48.395647 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b7cb1-2321-4cfc-99ed-ca34d7326a5e None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:48.406225 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b7cb1-2321-4cfc-99ed-ca34d7326a5e None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:48.463109 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56412b5-39d3-46fb-8830-8fd90c92ea59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:48.463708 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56412b5-39d3-46fb-8830-8fd90c92ea59 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:48.463986 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56412b5-39d3-46fb-8830-8fd90c92ea59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:48.464253 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56412b5-39d3-46fb-8830-8fd90c92ea59 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:48.464885 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56412b5-39d3-46fb-8830-8fd90c92ea59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:48.466812 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56412b5-39d3-46fb-8830-8fd90c92ea59 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:48.486915 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1141/2297] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:15:48 2022] POST /identity/v3/users => generated 296 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:15:48.490788 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f2316-2542-45fc-bf43-0cd53928a6c2 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:48.500956 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f2316-2542-45fc-bf43-0cd53928a6c2 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:48.505993 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95a6b1-ec92-41ed-a0eb-2f2fce4c1f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:48.506668 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95a6b1-ec92-41ed-a0eb-2f2fce4c1f67 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:48.506944 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95a6b1-ec92-41ed-a0eb-2f2fce4c1f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:48.507211 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e95a6b1-ec92-41ed-a0eb-2f2fce4c1f67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:48.507848 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e95a6b1-ec92-41ed-a0eb-2f2fce4c1f67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:48.509423 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e95a6b1-ec92-41ed-a0eb-2f2fce4c1f67 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:48.520336 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1157/2298] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:15:48.524294 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56412b5-39d3-46fb-8830-8fd90c92ea59 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:48.534621 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56412b5-39d3-46fb-8830-8fd90c92ea59 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:48.539321 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe933e-f46e-4729-b22b-3dee0780ef13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:48.539978 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe933e-f46e-4729-b22b-3dee0780ef13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:48.540261 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe933e-f46e-4729-b22b-3dee0780ef13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:48.540538 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfe933e-f46e-4729-b22b-3dee0780ef13 admin admin] PATH_INFO: `/v3/projects/cd3378ac27e2438d873d50a114d9a042/users/7fd3da30ae4f48d28ea472b61ef24654/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:48.557035 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe933e-f46e-4729-b22b-3dee0780ef13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd3378ac27e2438d873d50a114d9a042, user_id=7fd3da30ae4f48d28ea472b61ef24654, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:48.559848 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfe933e-f46e-4729-b22b-3dee0780ef13 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:48.574733 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1142/2299] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:15:48 2022] PUT /identity/v3/projects/cd3378ac27e2438d873d50a114d9a042/users/7fd3da30ae4f48d28ea472b61ef24654/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:48.579128 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95a6b1-ec92-41ed-a0eb-2f2fce4c1f67 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:48.590217 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e95a6b1-ec92-41ed-a0eb-2f2fce4c1f67 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:48.632746 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:48.654446 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b51a61f-a6c2-42b9-a139-085f5efa87ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:48.655081 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51a61f-a6c2-42b9-a139-085f5efa87ad admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:48.655359 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51a61f-a6c2-42b9-a139-085f5efa87ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:48.655629 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b51a61f-a6c2-42b9-a139-085f5efa87ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:48.656317 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b51a61f-a6c2-42b9-a139-085f5efa87ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:48.657932 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b51a61f-a6c2-42b9-a139-085f5efa87ad admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:48.672825 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1158/2300] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:48 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:15:48.673962 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe933e-f46e-4729-b22b-3dee0780ef13 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:48.684913 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfe933e-f46e-4729-b22b-3dee0780ef13 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:48.689703 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa188999-313e-4b65-b01d-f53109e176e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:48.690394 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa188999-313e-4b65-b01d-f53109e176e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:48.690673 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa188999-313e-4b65-b01d-f53109e176e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:48.691026 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa188999-313e-4b65-b01d-f53109e176e2 admin admin] PATH_INFO: `/v3/projects/cd3378ac27e2438d873d50a114d9a042/users/7fd3da30ae4f48d28ea472b61ef24654/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:48.692861 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa188999-313e-4b65-b01d-f53109e176e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd3378ac27e2438d873d50a114d9a042, user_id=7fd3da30ae4f48d28ea472b61ef24654, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:48.695843 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa188999-313e-4b65-b01d-f53109e176e2 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:48.706363 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1143/2301] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:15:48 2022] PUT /identity/v3/projects/cd3378ac27e2438d873d50a114d9a042/users/7fd3da30ae4f48d28ea472b61ef24654/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:15:48.717635 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd707b36-e07f-4d5b-bfc6-e198b389721a None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:48.717924 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd707b36-e07f-4d5b-bfc6-e198b389721a None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:48.718203 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd707b36-e07f-4d5b-bfc6-e198b389721a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:48.747841 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-bd707b36-e07f-4d5b-bfc6-e198b389721a None None] MFA Rules not processed for user `7fd3da30ae4f48d28ea472b61ef24654`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:48.749020 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-bd707b36-e07f-4d5b-bfc6-e198b389721a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:48.820343 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1159/2302] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:48 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:48.832901 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa188999-313e-4b65-b01d-f53109e176e2 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:48.834734 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-fa188999-313e-4b65-b01d-f53109e176e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:48.901981 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa188999-313e-4b65-b01d-f53109e176e2 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:48.906226 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4104cba9-e002-48e6-8b80-610d2a3ec3c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:48.906873 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4104cba9-e002-48e6-8b80-610d2a3ec3c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:48.907149 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4104cba9-e002-48e6-8b80-610d2a3ec3c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:48.907674 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4104cba9-e002-48e6-8b80-610d2a3ec3c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:48.918230 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4104cba9-e002-48e6-8b80-610d2a3ec3c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:48.919862 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4104cba9-e002-48e6-8b80-610d2a3ec3c8 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:48.930027 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1144/2303] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:15:48 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:15:48.970748 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd707b36-e07f-4d5b-bfc6-e198b389721a None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:48.972143 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-bd707b36-e07f-4d5b-bfc6-e198b389721a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:48.985542 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd707b36-e07f-4d5b-bfc6-e198b389721a None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:48.989201 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a73560-0e62-4788-930e-e3b98c274bb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:48.989796 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a73560-0e62-4788-930e-e3b98c274bb6 service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:48.990074 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a73560-0e62-4788-930e-e3b98c274bb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:48.990376 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a73560-0e62-4788-930e-e3b98c274bb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:49.047411 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a73560-0e62-4788-930e-e3b98c274bb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:49.048834 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a73560-0e62-4788-930e-e3b98c274bb6 service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:49.057654 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1160/2304] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:15:48 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:15:56.114630 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4104cba9-e002-48e6-8b80-610d2a3ec3c8 service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:56.159741 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:56.178764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4104cba9-e002-48e6-8b80-610d2a3ec3c8 service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:56.183207 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d2245-32b7-4466-96c1-a2c968a39dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:56.183821 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d2245-32b7-4466-96c1-a2c968a39dbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:56.184097 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d2245-32b7-4466-96c1-a2c968a39dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:56.184373 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497d2245-32b7-4466-96c1-a2c968a39dbb admin admin] PATH_INFO: `/v3/users/269f6ffd85cb446ab4716d7193a0b78f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:56.185524 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d2245-32b7-4466-96c1-a2c968a39dbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=269f6ffd85cb446ab4716d7193a0b78f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:56.187464 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497d2245-32b7-4466-96c1-a2c968a39dbb admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:56.232706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-497d2245-32b7-4466-96c1-a2c968a39dbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '269f6ffd85cb446ab4716d7193a0b78f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:56.245459 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-497d2245-32b7-4466-96c1-a2c968a39dbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '269f6ffd85cb446ab4716d7193a0b78f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:56.252634 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-497d2245-32b7-4466-96c1-a2c968a39dbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '269f6ffd85cb446ab4716d7193a0b78f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:56.253245 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-497d2245-32b7-4466-96c1-a2c968a39dbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '269f6ffd85cb446ab4716d7193a0b78f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:56.260307 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-497d2245-32b7-4466-96c1-a2c968a39dbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '269f6ffd85cb446ab4716d7193a0b78f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:56.266100 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1145/2305] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:15:56 2022] DELETE /identity/v3/users/269f6ffd85cb446ab4716d7193a0b78f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:15:56.270660 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cab65-8e4b-4ea4-9eb6-61f55e62c2e0 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:56.271215 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cab65-8e4b-4ea4-9eb6-61f55e62c2e0 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:56.271215 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18cab65-8e4b-4ea4-9eb6-61f55e62c2e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:56.290185 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-e18cab65-8e4b-4ea4-9eb6-61f55e62c2e0 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:56.291331 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-e18cab65-8e4b-4ea4-9eb6-61f55e62c2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:56.341422 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:56.350382 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1161/2306] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:56.360006 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d2245-32b7-4466-96c1-a2c968a39dbb admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:56.361402 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-497d2245-32b7-4466-96c1-a2c968a39dbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:56.420489 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497d2245-32b7-4466-96c1-a2c968a39dbb admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:56.424289 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75c943-f3ed-41d7-8a29-c72ad6c25f5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:56.424887 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75c943-f3ed-41d7-8a29-c72ad6c25f5d service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:56.425166 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75c943-f3ed-41d7-8a29-c72ad6c25f5d service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:56.425436 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c75c943-f3ed-41d7-8a29-c72ad6c25f5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:56.434829 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c75c943-f3ed-41d7-8a29-c72ad6c25f5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:56.436263 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c75c943-f3ed-41d7-8a29-c72ad6c25f5d service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:56.446612 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1146/2307] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:15:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:15:56.722527 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18cab65-8e4b-4ea4-9eb6-61f55e62c2e0 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:56.724613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-e18cab65-8e4b-4ea4-9eb6-61f55e62c2e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:56.741455 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18cab65-8e4b-4ea4-9eb6-61f55e62c2e0 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:56.746022 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ea6a7-a40d-4400-aa93-e04e8a65d77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:56.746720 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ea6a7-a40d-4400-aa93-e04e8a65d77f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:56.747017 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ea6a7-a40d-4400-aa93-e04e8a65d77f admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:56.747270 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3ea6a7-a40d-4400-aa93-e04e8a65d77f admin admin] PATH_INFO: `/v3/projects/d6de55c2c5d24a4d9f97763f184e7094` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:56.748377 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3ea6a7-a40d-4400-aa93-e04e8a65d77f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6de55c2c5d24a4d9f97763f184e7094)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:56.750379 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3ea6a7-a40d-4400-aa93-e04e8a65d77f admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:56.797253 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-3a3ea6a7-a40d-4400-aa93-e04e8a65d77f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6de55c2c5d24a4d9f97763f184e7094', 'request_id': 'req-3a3ea6a7-a40d-4400-aa93-e04e8a65d77f'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:56.804491 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-3a3ea6a7-a40d-4400-aa93-e04e8a65d77f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6de55c2c5d24a4d9f97763f184e7094', 'request_id': 'req-3a3ea6a7-a40d-4400-aa93-e04e8a65d77f'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:56.810274 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-3a3ea6a7-a40d-4400-aa93-e04e8a65d77f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6de55c2c5d24a4d9f97763f184e7094', 'request_id': 'req-3a3ea6a7-a40d-4400-aa93-e04e8a65d77f'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:56.821810 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-3a3ea6a7-a40d-4400-aa93-e04e8a65d77f admin admin] The token cache is being invalidate because project d6de55c2c5d24a4d9f97763f184e7094 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:15:56.822100 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-3a3ea6a7-a40d-4400-aa93-e04e8a65d77f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:56.822794 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-3a3ea6a7-a40d-4400-aa93-e04e8a65d77f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:56.824290 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1162/2308] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:15:56 2022] DELETE /identity/v3/projects/d6de55c2c5d24a4d9f97763f184e7094 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:56.828462 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75c943-f3ed-41d7-8a29-c72ad6c25f5d service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:56.838611 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c75c943-f3ed-41d7-8a29-c72ad6c25f5d service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:56.898526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b8ae40-cd18-4c62-831d-e82d93b99864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:56.899149 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8ae40-cd18-4c62-831d-e82d93b99864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:56.899424 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8ae40-cd18-4c62-831d-e82d93b99864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:56.899693 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b8ae40-cd18-4c62-831d-e82d93b99864 admin admin] PATH_INFO: `/v3/users/111ebeb244314a04968542a064d6f636` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:56.900801 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b8ae40-cd18-4c62-831d-e82d93b99864 admin admin] RBAC: Authorizing `identity:delete_user(user_id=111ebeb244314a04968542a064d6f636)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:56.902737 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b8ae40-cd18-4c62-831d-e82d93b99864 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:56.907828 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:56.947822 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d3b8ae40-cd18-4c62-831d-e82d93b99864 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '111ebeb244314a04968542a064d6f636'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:56.960631 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d3b8ae40-cd18-4c62-831d-e82d93b99864 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '111ebeb244314a04968542a064d6f636'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:56.967796 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d3b8ae40-cd18-4c62-831d-e82d93b99864 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '111ebeb244314a04968542a064d6f636'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:56.968437 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d3b8ae40-cd18-4c62-831d-e82d93b99864 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '111ebeb244314a04968542a064d6f636'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:56.975526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d3b8ae40-cd18-4c62-831d-e82d93b99864 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '111ebeb244314a04968542a064d6f636'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:56.981252 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1147/2309] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:15:56 2022] DELETE /identity/v3/users/111ebeb244314a04968542a064d6f636 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:57.249430 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ea6a7-a40d-4400-aa93-e04e8a65d77f admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:57.250888 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-3a3ea6a7-a40d-4400-aa93-e04e8a65d77f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:57.265679 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3ea6a7-a40d-4400-aa93-e04e8a65d77f admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:57.324267 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44054f9d-ffc6-4536-b5d5-391b8d4df788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:57.324893 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44054f9d-ffc6-4536-b5d5-391b8d4df788 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:57.325171 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44054f9d-ffc6-4536-b5d5-391b8d4df788 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:57.325699 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44054f9d-ffc6-4536-b5d5-391b8d4df788 admin admin] PATH_INFO: `/v3/projects/c42e7f1939de44dda1b805f4cc73ada6` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:57.326904 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44054f9d-ffc6-4536-b5d5-391b8d4df788 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c42e7f1939de44dda1b805f4cc73ada6)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:57.328788 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44054f9d-ffc6-4536-b5d5-391b8d4df788 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:57.375400 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-44054f9d-ffc6-4536-b5d5-391b8d4df788 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c42e7f1939de44dda1b805f4cc73ada6', 'request_id': 'req-44054f9d-ffc6-4536-b5d5-391b8d4df788'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:57.383305 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-44054f9d-ffc6-4536-b5d5-391b8d4df788 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c42e7f1939de44dda1b805f4cc73ada6', 'request_id': 'req-44054f9d-ffc6-4536-b5d5-391b8d4df788'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:57.389018 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-44054f9d-ffc6-4536-b5d5-391b8d4df788 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c42e7f1939de44dda1b805f4cc73ada6', 'request_id': 'req-44054f9d-ffc6-4536-b5d5-391b8d4df788'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:57.400000 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-44054f9d-ffc6-4536-b5d5-391b8d4df788 admin admin] The token cache is being invalidate because project c42e7f1939de44dda1b805f4cc73ada6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:15:57.400284 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-44054f9d-ffc6-4536-b5d5-391b8d4df788 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:57.400907 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-44054f9d-ffc6-4536-b5d5-391b8d4df788 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:57.402396 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1163/2310] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:15:57 2022] DELETE /identity/v3/projects/c42e7f1939de44dda1b805f4cc73ada6 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:57.406320 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b8ae40-cd18-4c62-831d-e82d93b99864 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:57.407730 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d3b8ae40-cd18-4c62-831d-e82d93b99864 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:57.421317 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b8ae40-cd18-4c62-831d-e82d93b99864 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:57.477043 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0766d4-d0d4-434b-acab-77151d67934e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:57.477678 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0766d4-d0d4-434b-acab-77151d67934e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:57.477960 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0766d4-d0d4-434b-acab-77151d67934e admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:57.478237 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0766d4-d0d4-434b-acab-77151d67934e admin admin] PATH_INFO: `/v3/users/7fd3da30ae4f48d28ea472b61ef24654` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:57.479383 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0766d4-d0d4-434b-acab-77151d67934e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fd3da30ae4f48d28ea472b61ef24654)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:57.481302 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0766d4-d0d4-434b-acab-77151d67934e admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:57.524778 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-da0766d4-d0d4-434b-acab-77151d67934e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fd3da30ae4f48d28ea472b61ef24654'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:57.537815 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-da0766d4-d0d4-434b-acab-77151d67934e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fd3da30ae4f48d28ea472b61ef24654'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:57.545893 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-da0766d4-d0d4-434b-acab-77151d67934e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fd3da30ae4f48d28ea472b61ef24654'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:57.546772 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-da0766d4-d0d4-434b-acab-77151d67934e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fd3da30ae4f48d28ea472b61ef24654'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:57.555077 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-da0766d4-d0d4-434b-acab-77151d67934e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fd3da30ae4f48d28ea472b61ef24654'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:57.561300 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1148/2311] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:15:57 2022] DELETE /identity/v3/users/7fd3da30ae4f48d28ea472b61ef24654 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:57.834733 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44054f9d-ffc6-4536-b5d5-391b8d4df788 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:57.836170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-44054f9d-ffc6-4536-b5d5-391b8d4df788 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:57.852185 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44054f9d-ffc6-4536-b5d5-391b8d4df788 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:57.862304 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:57.911215 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cf882c-3e53-48b1-8f88-1be880311a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:57.912399 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf882c-3e53-48b1-8f88-1be880311a78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:57.912700 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf882c-3e53-48b1-8f88-1be880311a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:57.912972 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cf882c-3e53-48b1-8f88-1be880311a78 admin admin] PATH_INFO: `/v3/projects/cd3378ac27e2438d873d50a114d9a042` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:57.914093 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cf882c-3e53-48b1-8f88-1be880311a78 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd3378ac27e2438d873d50a114d9a042)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:57.916002 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cf882c-3e53-48b1-8f88-1be880311a78 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:57.962154 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-20cf882c-3e53-48b1-8f88-1be880311a78 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd3378ac27e2438d873d50a114d9a042', 'request_id': 'req-20cf882c-3e53-48b1-8f88-1be880311a78'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:57.969249 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-20cf882c-3e53-48b1-8f88-1be880311a78 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd3378ac27e2438d873d50a114d9a042', 'request_id': 'req-20cf882c-3e53-48b1-8f88-1be880311a78'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:57.975580 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-20cf882c-3e53-48b1-8f88-1be880311a78 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd3378ac27e2438d873d50a114d9a042', 'request_id': 'req-20cf882c-3e53-48b1-8f88-1be880311a78'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:57.986458 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-20cf882c-3e53-48b1-8f88-1be880311a78 admin admin] The token cache is being invalidate because project cd3378ac27e2438d873d50a114d9a042 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:15:57.986747 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-20cf882c-3e53-48b1-8f88-1be880311a78 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:57.987380 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-20cf882c-3e53-48b1-8f88-1be880311a78 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:57.988850 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1164/2312] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:15:57 2022] DELETE /identity/v3/projects/cd3378ac27e2438d873d50a114d9a042 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:57.994909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26900445-8228-4f44-b3a0-7ac6ccef9b8c None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:57.995201 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26900445-8228-4f44-b3a0-7ac6ccef9b8c None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:57.995503 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26900445-8228-4f44-b3a0-7ac6ccef9b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:58.004466 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:58.017963 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-26900445-8228-4f44-b3a0-7ac6ccef9b8c None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:58.019930 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-26900445-8228-4f44-b3a0-7ac6ccef9b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:58.081085 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1149/2313] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:15:58.089865 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5528f33-2d25-4a1f-ab8f-d04906e54576 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:58.090151 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5528f33-2d25-4a1f-ab8f-d04906e54576 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:58.090458 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5528f33-2d25-4a1f-ab8f-d04906e54576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:58.110433 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-d5528f33-2d25-4a1f-ab8f-d04906e54576 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:58.111641 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-d5528f33-2d25-4a1f-ab8f-d04906e54576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:58.119482 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1165/2314] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:58.123582 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26900445-8228-4f44-b3a0-7ac6ccef9b8c None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:58.133085 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26900445-8228-4f44-b3a0-7ac6ccef9b8c None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:58.137732 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e549c5f0-ab3b-48c4-8f20-1033899a79e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:58.138365 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549c5f0-ab3b-48c4-8f20-1033899a79e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:58.138650 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549c5f0-ab3b-48c4-8f20-1033899a79e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:58.138927 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e549c5f0-ab3b-48c4-8f20-1033899a79e5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:58.139570 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e549c5f0-ab3b-48c4-8f20-1033899a79e5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:15:58.139875 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e549c5f0-ab3b-48c4-8f20-1033899a79e5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:58.141461 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e549c5f0-ab3b-48c4-8f20-1033899a79e5 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:58.153583 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1150/2315] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:15:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:15:58.158694 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5528f33-2d25-4a1f-ab8f-d04906e54576 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:58.171183 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5528f33-2d25-4a1f-ab8f-d04906e54576 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:58.175899 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a182a-3ff7-4857-a1d1-ccdbe2fb5d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:58.176503 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a182a-3ff7-4857-a1d1-ccdbe2fb5d1d admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:58.176780 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a182a-3ff7-4857-a1d1-ccdbe2fb5d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:58.177052 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a182a-3ff7-4857-a1d1-ccdbe2fb5d1d admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:58.177695 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a182a-3ff7-4857-a1d1-ccdbe2fb5d1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:58.179621 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a182a-3ff7-4857-a1d1-ccdbe2fb5d1d admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:58.206592 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1166/2316] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:15:58 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:15:58.211727 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e549c5f0-ab3b-48c4-8f20-1033899a79e5 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:58.221532 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e549c5f0-ab3b-48c4-8f20-1033899a79e5 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:58.279532 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb1998c-662b-4544-8890-a2fcd3e902ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:58.280139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb1998c-662b-4544-8890-a2fcd3e902ae admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:58.280420 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb1998c-662b-4544-8890-a2fcd3e902ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:58.280698 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb1998c-662b-4544-8890-a2fcd3e902ae admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:58.281342 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb1998c-662b-4544-8890-a2fcd3e902ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:58.283303 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb1998c-662b-4544-8890-a2fcd3e902ae admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:58.305555 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1151/2317] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:15:58 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:15:58.309755 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a182a-3ff7-4857-a1d1-ccdbe2fb5d1d admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:58.320639 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a182a-3ff7-4857-a1d1-ccdbe2fb5d1d admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:58.325498 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb02b3-a852-4104-b199-2875162b908a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:58.326106 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb02b3-a852-4104-b199-2875162b908a admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:58.326435 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb02b3-a852-4104-b199-2875162b908a admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:58.326699 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bb02b3-a852-4104-b199-2875162b908a admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:58.327342 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bb02b3-a852-4104-b199-2875162b908a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:58.328920 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bb02b3-a852-4104-b199-2875162b908a admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:58.339357 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1167/2318] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:15:58.343105 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb1998c-662b-4544-8890-a2fcd3e902ae admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:58.352884 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb1998c-662b-4544-8890-a2fcd3e902ae admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:58.357476 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2c673a-88a5-4e0f-a71c-64ed25d2d437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:58.358064 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c673a-88a5-4e0f-a71c-64ed25d2d437 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:58.358374 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c673a-88a5-4e0f-a71c-64ed25d2d437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:58.358653 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2c673a-88a5-4e0f-a71c-64ed25d2d437 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:58.359294 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c673a-88a5-4e0f-a71c-64ed25d2d437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:58.360876 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2c673a-88a5-4e0f-a71c-64ed25d2d437 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:58.371406 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1152/2319] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:15:58.375271 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb02b3-a852-4104-b199-2875162b908a admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:58.386062 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bb02b3-a852-4104-b199-2875162b908a admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:58.390767 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5927ed7-9279-40a9-85ab-7507944353e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:58.391414 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5927ed7-9279-40a9-85ab-7507944353e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:58.391692 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5927ed7-9279-40a9-85ab-7507944353e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:58.391978 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5927ed7-9279-40a9-85ab-7507944353e0 admin admin] PATH_INFO: `/v3/projects/8b8798c4802f4e249aa01a33bc67e04c/users/219786c92bda4bd88243a9fccee7f71d/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:58.406939 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5927ed7-9279-40a9-85ab-7507944353e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b8798c4802f4e249aa01a33bc67e04c, user_id=219786c92bda4bd88243a9fccee7f71d, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:58.409586 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5927ed7-9279-40a9-85ab-7507944353e0 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:58.422320 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1168/2320] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:15:58 2022] PUT /identity/v3/projects/8b8798c4802f4e249aa01a33bc67e04c/users/219786c92bda4bd88243a9fccee7f71d/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:15:58.433535 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667fbf9c-ec5c-4d63-ae5d-fcc133bfbbbc None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:58.434038 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667fbf9c-ec5c-4d63-ae5d-fcc133bfbbbc None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:58.434136 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667fbf9c-ec5c-4d63-ae5d-fcc133bfbbbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:58.462978 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-667fbf9c-ec5c-4d63-ae5d-fcc133bfbbbc None None] MFA Rules not processed for user `219786c92bda4bd88243a9fccee7f71d`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:58.464186 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-667fbf9c-ec5c-4d63-ae5d-fcc133bfbbbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:58.524847 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:58.529511 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1153/2321] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:58.533352 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5927ed7-9279-40a9-85ab-7507944353e0 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:58.575695 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:58.595954 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5927ed7-9279-40a9-85ab-7507944353e0 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:58.600519 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fafceb-7092-44c7-91fb-ff14725e3728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:58.601110 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fafceb-7092-44c7-91fb-ff14725e3728 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:58.601392 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fafceb-7092-44c7-91fb-ff14725e3728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:58.601663 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fafceb-7092-44c7-91fb-ff14725e3728 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:58.602350 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fafceb-7092-44c7-91fb-ff14725e3728 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:58.604246 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0fafceb-7092-44c7-91fb-ff14725e3728 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:58.629537 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1169/2322] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:15:58 2022] POST /identity/v3/projects => generated 365 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:15:58.633999 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667fbf9c-ec5c-4d63-ae5d-fcc133bfbbbc None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:58.644466 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667fbf9c-ec5c-4d63-ae5d-fcc133bfbbbc None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:58.679712 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fafceb-7092-44c7-91fb-ff14725e3728 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:58.681392 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-e0fafceb-7092-44c7-91fb-ff14725e3728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:58.703460 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9546516-486c-4ef5-b5b2-e6b1a042c7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:58.704070 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9546516-486c-4ef5-b5b2-e6b1a042c7a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:58.704349 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9546516-486c-4ef5-b5b2-e6b1a042c7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:58.704622 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9546516-486c-4ef5-b5b2-e6b1a042c7a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:58.705261 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9546516-486c-4ef5-b5b2-e6b1a042c7a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:58.707215 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9546516-486c-4ef5-b5b2-e6b1a042c7a2 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:58.729166 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1154/2323] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:15:58 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:15:58.733013 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9546516-486c-4ef5-b5b2-e6b1a042c7a2 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:58.743277 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9546516-486c-4ef5-b5b2-e6b1a042c7a2 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:58.748618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fafceb-7092-44c7-91fb-ff14725e3728 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:58.748892 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc23638c-7fda-4f36-8fd2-1ae08352395b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:58.749512 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23638c-7fda-4f36-8fd2-1ae08352395b admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:58.749814 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23638c-7fda-4f36-8fd2-1ae08352395b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:58.750075 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc23638c-7fda-4f36-8fd2-1ae08352395b admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:58.750749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc23638c-7fda-4f36-8fd2-1ae08352395b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:58.752332 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc23638c-7fda-4f36-8fd2-1ae08352395b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:58.753416 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9188c77-4a5b-40fc-bac0-377f488a1f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:58.754039 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9188c77-4a5b-40fc-bac0-377f488a1f1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:58.754348 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9188c77-4a5b-40fc-bac0-377f488a1f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:58.754627 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9188c77-4a5b-40fc-bac0-377f488a1f1f admin admin] PATH_INFO: `/v3/users/353f35382f23462ea44d595475dd5a60` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:58.755838 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9188c77-4a5b-40fc-bac0-377f488a1f1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=353f35382f23462ea44d595475dd5a60)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:58.758432 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9188c77-4a5b-40fc-bac0-377f488a1f1f admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:58.763199 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1155/2324] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 20:15:58.767343 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc23638c-7fda-4f36-8fd2-1ae08352395b admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:58.777088 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc23638c-7fda-4f36-8fd2-1ae08352395b admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:58.782108 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade305af-1fe9-4db7-8370-f89dfcfa71b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:58.782907 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade305af-1fe9-4db7-8370-f89dfcfa71b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:58.783226 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade305af-1fe9-4db7-8370-f89dfcfa71b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:58.783567 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade305af-1fe9-4db7-8370-f89dfcfa71b6 admin admin] PATH_INFO: `/v3/projects/740697ca9b09496396cc8a471eb3451d/users/019d5a909f7b434cb5e0ae59d61f2aa7/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:58.799414 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade305af-1fe9-4db7-8370-f89dfcfa71b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=740697ca9b09496396cc8a471eb3451d, user_id=019d5a909f7b434cb5e0ae59d61f2aa7, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:58.802060 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade305af-1fe9-4db7-8370-f89dfcfa71b6 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:58.806984 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-a9188c77-4a5b-40fc-bac0-377f488a1f1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '353f35382f23462ea44d595475dd5a60'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:58.816081 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1156/2325] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:15:58 2022] PUT /identity/v3/projects/740697ca9b09496396cc8a471eb3451d/users/019d5a909f7b434cb5e0ae59d61f2aa7/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:15:58.820482 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade305af-1fe9-4db7-8370-f89dfcfa71b6 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:58.821773 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-a9188c77-4a5b-40fc-bac0-377f488a1f1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '353f35382f23462ea44d595475dd5a60'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:58.829286 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-a9188c77-4a5b-40fc-bac0-377f488a1f1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '353f35382f23462ea44d595475dd5a60'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:58.837252 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-a9188c77-4a5b-40fc-bac0-377f488a1f1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '353f35382f23462ea44d595475dd5a60'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:58.837950 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-a9188c77-4a5b-40fc-bac0-377f488a1f1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '353f35382f23462ea44d595475dd5a60'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:58.844094 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1170/2326] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:15:58 2022] DELETE /identity/v3/users/353f35382f23462ea44d595475dd5a60 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:15:58.848197 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d623a8-b2e3-4ff8-b14b-e68bf00eba72 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:58.848524 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d623a8-b2e3-4ff8-b14b-e68bf00eba72 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:58.848870 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d623a8-b2e3-4ff8-b14b-e68bf00eba72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:58.868237 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-93d623a8-b2e3-4ff8-b14b-e68bf00eba72 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:58.869452 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-93d623a8-b2e3-4ff8-b14b-e68bf00eba72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:58.885167 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade305af-1fe9-4db7-8370-f89dfcfa71b6 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:58.910515 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:58.932211 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1171/2327] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:15:58.941975 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d623a8-b2e3-4ff8-b14b-e68bf00eba72 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:58.942337 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b3f652-ee99-411e-8bb6-e4a7be7dc172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:58.942955 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3f652-ee99-411e-8bb6-e4a7be7dc172 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:58.943234 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3f652-ee99-411e-8bb6-e4a7be7dc172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:58.943453 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-93d623a8-b2e3-4ff8-b14b-e68bf00eba72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:58.943563 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b3f652-ee99-411e-8bb6-e4a7be7dc172 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:58.944139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b3f652-ee99-411e-8bb6-e4a7be7dc172 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:58.945730 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b3f652-ee99-411e-8bb6-e4a7be7dc172 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:58.948239 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:58.958293 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1157/2328] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:15:58.961867 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b3f652-ee99-411e-8bb6-e4a7be7dc172 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:58.963591 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-12b3f652-ee99-411e-8bb6-e4a7be7dc172 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:58.976276 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b3f652-ee99-411e-8bb6-e4a7be7dc172 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:58.981046 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2c29ea-7fb9-42da-908c-ddf6e94d663d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:58.981687 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c29ea-7fb9-42da-908c-ddf6e94d663d admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:58.981966 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c29ea-7fb9-42da-908c-ddf6e94d663d admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:58.982240 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2c29ea-7fb9-42da-908c-ddf6e94d663d admin admin] PATH_INFO: `/v3/projects/740697ca9b09496396cc8a471eb3451d/users/019d5a909f7b434cb5e0ae59d61f2aa7/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:58.984173 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2c29ea-7fb9-42da-908c-ddf6e94d663d admin admin] RBAC: Authorizing `identity:create_grant(project_id=740697ca9b09496396cc8a471eb3451d, user_id=019d5a909f7b434cb5e0ae59d61f2aa7, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:58.986990 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2c29ea-7fb9-42da-908c-ddf6e94d663d admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:58.997711 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1158/2329] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:15:58 2022] PUT /identity/v3/projects/740697ca9b09496396cc8a471eb3451d/users/019d5a909f7b434cb5e0ae59d61f2aa7/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:15:59.007862 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d623a8-b2e3-4ff8-b14b-e68bf00eba72 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:59.008738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e359d-d958-4e84-8d7d-3d53b013cb44 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:59.009145 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e359d-d958-4e84-8d7d-3d53b013cb44 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:59.009433 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44e359d-d958-4e84-8d7d-3d53b013cb44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:59.037302 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-e44e359d-d958-4e84-8d7d-3d53b013cb44 None None] MFA Rules not processed for user `019d5a909f7b434cb5e0ae59d61f2aa7`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:59.039114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e44e359d-d958-4e84-8d7d-3d53b013cb44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:59.063689 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a72ab-a1ca-4274-ac60-83779051f429 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:59.064295 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a72ab-a1ca-4274-ac60-83779051f429 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:59.064570 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a72ab-a1ca-4274-ac60-83779051f429 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:59.064835 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14a72ab-a1ca-4274-ac60-83779051f429 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:59.115398 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1159/2330] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:59 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:15:59.127616 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e359d-d958-4e84-8d7d-3d53b013cb44 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:59.128969 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e44e359d-d958-4e84-8d7d-3d53b013cb44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:59.129280 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a72ab-a1ca-4274-ac60-83779051f429 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:59.130770 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14a72ab-a1ca-4274-ac60-83779051f429 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:59.140196 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1172/2331] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:15:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 20:15:59.188284 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44e359d-d958-4e84-8d7d-3d53b013cb44 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:59.192005 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a23a1a-6634-4ac2-b281-95027db18667 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:59.192649 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a23a1a-6634-4ac2-b281-95027db18667 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:59.192933 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a23a1a-6634-4ac2-b281-95027db18667 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:59.193210 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a23a1a-6634-4ac2-b281-95027db18667 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:59.202909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a23a1a-6634-4ac2-b281-95027db18667 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:59.204364 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a23a1a-6634-4ac2-b281-95027db18667 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:59.214159 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1160/2332] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:15:59 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 20:15:59.299817 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a72ab-a1ca-4274-ac60-83779051f429 service neutron] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:59.311126 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14a72ab-a1ca-4274-ac60-83779051f429 service neutron] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:59.314822 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b78f8-f692-41a0-8883-15b9aa0c63bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:59.315428 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b78f8-f692-41a0-8883-15b9aa0c63bb service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:59.315707 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b78f8-f692-41a0-8883-15b9aa0c63bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:59.316034 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b78f8-f692-41a0-8883-15b9aa0c63bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:59.317774 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-f25b78f8-f692-41a0-8883-15b9aa0c63bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:59.368355 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:59.380157 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25b78f8-f692-41a0-8883-15b9aa0c63bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:59.381630 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25b78f8-f692-41a0-8883-15b9aa0c63bb service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:59.391359 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1173/2333] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:15:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:15:59.416134 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a23a1a-6634-4ac2-b281-95027db18667 service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:59.417703 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f0a23a1a-6634-4ac2-b281-95027db18667 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:59.433598 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a23a1a-6634-4ac2-b281-95027db18667 service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:59.439155 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c85188-8c2e-4157-8579-9a163f45e94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:59.442234 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c85188-8c2e-4157-8579-9a163f45e94f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:59.442589 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c85188-8c2e-4157-8579-9a163f45e94f admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:59.442876 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c85188-8c2e-4157-8579-9a163f45e94f admin admin] PATH_INFO: `/v3/projects/6f33ce26965d45beb8990a8c6affac40` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:59.444777 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c85188-8c2e-4157-8579-9a163f45e94f admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f33ce26965d45beb8990a8c6affac40)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:59.448304 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c85188-8c2e-4157-8579-9a163f45e94f admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:59.498022 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-c7c85188-8c2e-4157-8579-9a163f45e94f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f33ce26965d45beb8990a8c6affac40', 'request_id': 'req-c7c85188-8c2e-4157-8579-9a163f45e94f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:59.505638 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-c7c85188-8c2e-4157-8579-9a163f45e94f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f33ce26965d45beb8990a8c6affac40', 'request_id': 'req-c7c85188-8c2e-4157-8579-9a163f45e94f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:59.511775 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-c7c85188-8c2e-4157-8579-9a163f45e94f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f33ce26965d45beb8990a8c6affac40', 'request_id': 'req-c7c85188-8c2e-4157-8579-9a163f45e94f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:59.513798 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:15:59.525162 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-c7c85188-8c2e-4157-8579-9a163f45e94f admin admin] The token cache is being invalidate because project 6f33ce26965d45beb8990a8c6affac40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:15:59.525691 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-c7c85188-8c2e-4157-8579-9a163f45e94f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:59.526870 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-c7c85188-8c2e-4157-8579-9a163f45e94f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:15:59.528853 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1161/2334] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:15:59 2022] DELETE /identity/v3/projects/6f33ce26965d45beb8990a8c6affac40 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:59.533460 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ad508-bde1-44ed-a675-aa36b0296d25 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:59.533755 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ad508-bde1-44ed-a675-aa36b0296d25 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:59.534029 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944ad508-bde1-44ed-a675-aa36b0296d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:59.555762 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-944ad508-bde1-44ed-a675-aa36b0296d25 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:59.557208 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-944ad508-bde1-44ed-a675-aa36b0296d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:59.622474 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1174/2335] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:15:59.630157 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4d1c1-d7bc-4238-a62e-439006bec86e None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:59.630678 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4d1c1-d7bc-4238-a62e-439006bec86e None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:59.631023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae4d1c1-d7bc-4238-a62e-439006bec86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:59.653790 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-3ae4d1c1-d7bc-4238-a62e-439006bec86e None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:59.654974 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3ae4d1c1-d7bc-4238-a62e-439006bec86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:15:59.662974 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1162/2336] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:15:59.667315 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944ad508-bde1-44ed-a675-aa36b0296d25 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:59.677212 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944ad508-bde1-44ed-a675-aa36b0296d25 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:59.682094 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c42644-8571-4497-a012-ebaa07043e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:59.683123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c42644-8571-4497-a012-ebaa07043e33 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:59.683427 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c42644-8571-4497-a012-ebaa07043e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:59.683669 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c42644-8571-4497-a012-ebaa07043e33 admin admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:59.686295 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c42644-8571-4497-a012-ebaa07043e33 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:15:59.686295 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c42644-8571-4497-a012-ebaa07043e33 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:59.686295 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c42644-8571-4497-a012-ebaa07043e33 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:59.699618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1175/2337] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:15:59.706210 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4d1c1-d7bc-4238-a62e-439006bec86e None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:59.716021 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae4d1c1-d7bc-4238-a62e-439006bec86e None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:59.720643 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21fab0c-6b99-401a-92da-b4941aa247c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:59.721240 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fab0c-6b99-401a-92da-b4941aa247c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:59.721519 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fab0c-6b99-401a-92da-b4941aa247c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:59.721790 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fab0c-6b99-401a-92da-b4941aa247c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:59.722472 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21fab0c-6b99-401a-92da-b4941aa247c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:59.724368 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21fab0c-6b99-401a-92da-b4941aa247c3 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:59.751700 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1163/2338] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:15:59 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:15:59.755774 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c42644-8571-4497-a012-ebaa07043e33 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:59.766177 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c42644-8571-4497-a012-ebaa07043e33 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:59.821440 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc86e9d-c820-45e0-ab13-b5ce302ae1fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:59.822033 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc86e9d-c820-45e0-ab13-b5ce302ae1fe admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:59.822337 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc86e9d-c820-45e0-ab13-b5ce302ae1fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:59.822607 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc86e9d-c820-45e0-ab13-b5ce302ae1fe admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:59.823246 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc86e9d-c820-45e0-ab13-b5ce302ae1fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:59.825143 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc86e9d-c820-45e0-ab13-b5ce302ae1fe admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:59.846076 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1176/2339] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:15:59 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:15:59.850469 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21fab0c-6b99-401a-92da-b4941aa247c3 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:59.860427 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21fab0c-6b99-401a-92da-b4941aa247c3 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:59.865072 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d06e7b-3e13-4951-90bd-9619ad6215e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:59.865684 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d06e7b-3e13-4951-90bd-9619ad6215e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:59.865960 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d06e7b-3e13-4951-90bd-9619ad6215e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:59.866229 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d06e7b-3e13-4951-90bd-9619ad6215e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:59.866891 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d06e7b-3e13-4951-90bd-9619ad6215e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:59.868478 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d06e7b-3e13-4951-90bd-9619ad6215e4 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:59.878374 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1164/2340] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:15:59.882934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc86e9d-c820-45e0-ab13-b5ce302ae1fe admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:59.892899 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc86e9d-c820-45e0-ab13-b5ce302ae1fe admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:59.898060 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd7c778-8bdb-43ca-9231-876c3c06e111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:59.898674 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7c778-8bdb-43ca-9231-876c3c06e111 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:59.898950 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7c778-8bdb-43ca-9231-876c3c06e111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:59.899218 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd7c778-8bdb-43ca-9231-876c3c06e111 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:59.899857 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd7c778-8bdb-43ca-9231-876c3c06e111 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:59.901433 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd7c778-8bdb-43ca-9231-876c3c06e111 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:59.911985 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1177/2341] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:15:59.915685 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d06e7b-3e13-4951-90bd-9619ad6215e4 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:15:59.925417 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d06e7b-3e13-4951-90bd-9619ad6215e4 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:15:59.930006 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be31a83-8dee-425a-964b-c89a5f600acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:15:59.930683 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be31a83-8dee-425a-964b-c89a5f600acf admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:59.930965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be31a83-8dee-425a-964b-c89a5f600acf admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:59.931242 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be31a83-8dee-425a-964b-c89a5f600acf admin admin] PATH_INFO: `/v3/projects/0df26e575de34eda9d416de65ec13036/users/52c576b9f49b4900ac5ba707aba78c27/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:59.946151 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be31a83-8dee-425a-964b-c89a5f600acf admin admin] RBAC: Authorizing `identity:create_grant(project_id=0df26e575de34eda9d416de65ec13036, user_id=52c576b9f49b4900ac5ba707aba78c27, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:15:59.948848 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be31a83-8dee-425a-964b-c89a5f600acf admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:15:59.962229 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1165/2342] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:15:59 2022] PUT /identity/v3/projects/0df26e575de34eda9d416de65ec13036/users/52c576b9f49b4900ac5ba707aba78c27/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:15:59.967417 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64ba5e-d871-43c6-964c-8fd9fcd868d4 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:15:59.967709 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64ba5e-d871-43c6-964c-8fd9fcd868d4 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:15:59.967983 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db64ba5e-d871-43c6-964c-8fd9fcd868d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:15:59.986502 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-db64ba5e-d871-43c6-964c-8fd9fcd868d4 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:15:59.987730 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-db64ba5e-d871-43c6-964c-8fd9fcd868d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:00.036586 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:16:00.048749 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1178/2343] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:15:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:16:00.058873 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be31a83-8dee-425a-964b-c89a5f600acf admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:00.060579 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-8be31a83-8dee-425a-964b-c89a5f600acf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:00.118534 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be31a83-8dee-425a-964b-c89a5f600acf admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:00.122165 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ea931-d136-4aec-81ee-e6d6abff1592 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:00.122793 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ea931-d136-4aec-81ee-e6d6abff1592 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:00.123071 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ea931-d136-4aec-81ee-e6d6abff1592 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:00.123344 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2ea931-d136-4aec-81ee-e6d6abff1592 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:00.133184 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2ea931-d136-4aec-81ee-e6d6abff1592 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:00.134602 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2ea931-d136-4aec-81ee-e6d6abff1592 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:00.143844 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1166/2344] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:16:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:16:00.736411 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef58bb-ecec-4952-8332-f4f758a4257b None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:00.736888 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef58bb-ecec-4952-8332-f4f758a4257b None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:00.737178 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef58bb-ecec-4952-8332-f4f758a4257b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:00.772527 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-e6ef58bb-ecec-4952-8332-f4f758a4257b None None] MFA Rules not processed for user `52c576b9f49b4900ac5ba707aba78c27`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:16:00.773745 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-e6ef58bb-ecec-4952-8332-f4f758a4257b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:00.869409 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1179/2345] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:16:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:16:00.878386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ea931-d136-4aec-81ee-e6d6abff1592 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:00.881894 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-6b2ea931-d136-4aec-81ee-e6d6abff1592 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:00.953422 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2ea931-d136-4aec-81ee-e6d6abff1592 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:00.957786 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07648ba-d094-4578-a31a-df90aeaff18a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:00.958513 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07648ba-d094-4578-a31a-df90aeaff18a service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:00.958799 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07648ba-d094-4578-a31a-df90aeaff18a service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:00.959074 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07648ba-d094-4578-a31a-df90aeaff18a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:00.968207 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07648ba-d094-4578-a31a-df90aeaff18a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:00.969657 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07648ba-d094-4578-a31a-df90aeaff18a service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:00.979233 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1167/2346] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:16:00 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:16:02.352033 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91186c8e-00f9-4dd6-b7b3-e6d286f75420 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:02.352564 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91186c8e-00f9-4dd6-b7b3-e6d286f75420 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:02.352564 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91186c8e-00f9-4dd6-b7b3-e6d286f75420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:02.371979 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-91186c8e-00f9-4dd6-b7b3-e6d286f75420 None None] MFA Rules not processed for user `52c576b9f49b4900ac5ba707aba78c27`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:16:02.373182 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-91186c8e-00f9-4dd6-b7b3-e6d286f75420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:02.380618 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1180/2347] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:16:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:16:04.226965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07648ba-d094-4578-a31a-df90aeaff18a service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:04.228433 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e07648ba-d094-4578-a31a-df90aeaff18a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:04.242757 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07648ba-d094-4578-a31a-df90aeaff18a service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:04.247350 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c665e-a828-44e7-b0cb-9e66fbb503b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:04.247980 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c665e-a828-44e7-b0cb-9e66fbb503b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:04.248268 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c665e-a828-44e7-b0cb-9e66fbb503b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:04.248546 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3c665e-a828-44e7-b0cb-9e66fbb503b5 admin admin] PATH_INFO: `/v3/users/219786c92bda4bd88243a9fccee7f71d` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:04.249761 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3c665e-a828-44e7-b0cb-9e66fbb503b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=219786c92bda4bd88243a9fccee7f71d)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:04.251806 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3c665e-a828-44e7-b0cb-9e66fbb503b5 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:04.288384 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:16:04.302647 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9b3c665e-a828-44e7-b0cb-9e66fbb503b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '219786c92bda4bd88243a9fccee7f71d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:04.315898 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9b3c665e-a828-44e7-b0cb-9e66fbb503b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '219786c92bda4bd88243a9fccee7f71d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:04.324450 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9b3c665e-a828-44e7-b0cb-9e66fbb503b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '219786c92bda4bd88243a9fccee7f71d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:04.325099 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9b3c665e-a828-44e7-b0cb-9e66fbb503b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '219786c92bda4bd88243a9fccee7f71d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:04.332962 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9b3c665e-a828-44e7-b0cb-9e66fbb503b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '219786c92bda4bd88243a9fccee7f71d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:04.340285 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1168/2348] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:16:04 2022] DELETE /identity/v3/users/219786c92bda4bd88243a9fccee7f71d => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:16:04.344515 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e49133-d482-4b06-9810-2120dac6ce6f None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:04.345000 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e49133-d482-4b06-9810-2120dac6ce6f None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:04.345098 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e49133-d482-4b06-9810-2120dac6ce6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:04.364308 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-97e49133-d482-4b06-9810-2120dac6ce6f None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:16:04.365728 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-97e49133-d482-4b06-9810-2120dac6ce6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:04.426274 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1181/2349] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:16:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:16:04.436197 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c665e-a828-44e7-b0cb-9e66fbb503b5 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:04.437738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9b3c665e-a828-44e7-b0cb-9e66fbb503b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:04.498748 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3c665e-a828-44e7-b0cb-9e66fbb503b5 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:04.502431 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572bc3b7-e35b-41f2-9051-defd3b93d1ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:04.503042 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572bc3b7-e35b-41f2-9051-defd3b93d1ba service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:04.503323 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572bc3b7-e35b-41f2-9051-defd3b93d1ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:04.503597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572bc3b7-e35b-41f2-9051-defd3b93d1ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:04.513296 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572bc3b7-e35b-41f2-9051-defd3b93d1ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:04.514758 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572bc3b7-e35b-41f2-9051-defd3b93d1ba service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:04.525333 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1169/2350] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:16:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:16:04.772781 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e49133-d482-4b06-9810-2120dac6ce6f None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:04.774640 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-97e49133-d482-4b06-9810-2120dac6ce6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:04.786915 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e49133-d482-4b06-9810-2120dac6ce6f None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:04.790564 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be8934-8fa0-43dc-8791-ef7813aaa953 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:04.791310 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be8934-8fa0-43dc-8791-ef7813aaa953 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:04.791595 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be8934-8fa0-43dc-8791-ef7813aaa953 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:04.791862 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be8934-8fa0-43dc-8791-ef7813aaa953 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:04.793441 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572bc3b7-e35b-41f2-9051-defd3b93d1ba service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:04.793937 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-d1be8934-8fa0-43dc-8791-ef7813aaa953 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:04.795031 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-572bc3b7-e35b-41f2-9051-defd3b93d1ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:04.804361 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1be8934-8fa0-43dc-8791-ef7813aaa953 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:04.805780 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1be8934-8fa0-43dc-8791-ef7813aaa953 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:04.814735 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572bc3b7-e35b-41f2-9051-defd3b93d1ba service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:04.814871 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1182/2351] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:16:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:16:04.820819 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e8529f-d238-430a-994a-296cea59285f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:04.821459 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8529f-d238-430a-994a-296cea59285f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:04.821745 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8529f-d238-430a-994a-296cea59285f admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:04.822475 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8529f-d238-430a-994a-296cea59285f admin admin] PATH_INFO: `/v3/projects/8b8798c4802f4e249aa01a33bc67e04c` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:04.823389 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e8529f-d238-430a-994a-296cea59285f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b8798c4802f4e249aa01a33bc67e04c)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:04.825899 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e8529f-d238-430a-994a-296cea59285f admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:04.878116 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-64e8529f-d238-430a-994a-296cea59285f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b8798c4802f4e249aa01a33bc67e04c', 'request_id': 'req-64e8529f-d238-430a-994a-296cea59285f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:04.887616 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-64e8529f-d238-430a-994a-296cea59285f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b8798c4802f4e249aa01a33bc67e04c', 'request_id': 'req-64e8529f-d238-430a-994a-296cea59285f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:04.893999 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-64e8529f-d238-430a-994a-296cea59285f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b8798c4802f4e249aa01a33bc67e04c', 'request_id': 'req-64e8529f-d238-430a-994a-296cea59285f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:04.905802 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-64e8529f-d238-430a-994a-296cea59285f admin admin] The token cache is being invalidate because project 8b8798c4802f4e249aa01a33bc67e04c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:16:04.906102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-64e8529f-d238-430a-994a-296cea59285f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:04.907308 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-64e8529f-d238-430a-994a-296cea59285f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:04.908826 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1170/2352] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:16:04 2022] DELETE /identity/v3/projects/8b8798c4802f4e249aa01a33bc67e04c => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:16:04.913270 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be8934-8fa0-43dc-8791-ef7813aaa953 service neutron] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:04.924238 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be8934-8fa0-43dc-8791-ef7813aaa953 service neutron] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:04.983572 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48826d15-bfb2-4fb5-90ba-4bf72771fb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:04.984205 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48826d15-bfb2-4fb5-90ba-4bf72771fb2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:04.984480 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48826d15-bfb2-4fb5-90ba-4bf72771fb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:04.984750 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48826d15-bfb2-4fb5-90ba-4bf72771fb2f admin admin] PATH_INFO: `/v3/users/019d5a909f7b434cb5e0ae59d61f2aa7` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:04.986007 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48826d15-bfb2-4fb5-90ba-4bf72771fb2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=019d5a909f7b434cb5e0ae59d61f2aa7)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:04.988129 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48826d15-bfb2-4fb5-90ba-4bf72771fb2f admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:04.994277 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=100 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:16:05.041735 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-48826d15-bfb2-4fb5-90ba-4bf72771fb2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '019d5a909f7b434cb5e0ae59d61f2aa7'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:05.054102 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-48826d15-bfb2-4fb5-90ba-4bf72771fb2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '019d5a909f7b434cb5e0ae59d61f2aa7'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:05.060940 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-48826d15-bfb2-4fb5-90ba-4bf72771fb2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '019d5a909f7b434cb5e0ae59d61f2aa7'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:05.069048 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-48826d15-bfb2-4fb5-90ba-4bf72771fb2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '019d5a909f7b434cb5e0ae59d61f2aa7'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:05.069703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-48826d15-bfb2-4fb5-90ba-4bf72771fb2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '019d5a909f7b434cb5e0ae59d61f2aa7'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:05.076459 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1183/2353] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:16:04 2022] DELETE /identity/v3/users/019d5a909f7b434cb5e0ae59d61f2aa7 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:16:05.365370 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e8529f-d238-430a-994a-296cea59285f admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:05.367588 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-64e8529f-d238-430a-994a-296cea59285f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:05.391865 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e8529f-d238-430a-994a-296cea59285f admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:05.430178 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:16:05.468650 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff14b4-6763-4431-ba3c-ed92b3ce9a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:05.469294 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff14b4-6763-4431-ba3c-ed92b3ce9a1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:05.469578 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff14b4-6763-4431-ba3c-ed92b3ce9a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:05.469856 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff14b4-6763-4431-ba3c-ed92b3ce9a1d admin admin] PATH_INFO: `/v3/projects/740697ca9b09496396cc8a471eb3451d` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:05.471030 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ff14b4-6763-4431-ba3c-ed92b3ce9a1d admin admin] RBAC: Authorizing `identity:delete_project(project_id=740697ca9b09496396cc8a471eb3451d)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:05.472990 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ff14b4-6763-4431-ba3c-ed92b3ce9a1d admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:05.520812 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e4ff14b4-6763-4431-ba3c-ed92b3ce9a1d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '740697ca9b09496396cc8a471eb3451d', 'request_id': 'req-e4ff14b4-6763-4431-ba3c-ed92b3ce9a1d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:05.529171 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e4ff14b4-6763-4431-ba3c-ed92b3ce9a1d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '740697ca9b09496396cc8a471eb3451d', 'request_id': 'req-e4ff14b4-6763-4431-ba3c-ed92b3ce9a1d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:05.535595 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e4ff14b4-6763-4431-ba3c-ed92b3ce9a1d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '740697ca9b09496396cc8a471eb3451d', 'request_id': 'req-e4ff14b4-6763-4431-ba3c-ed92b3ce9a1d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:05.549974 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e4ff14b4-6763-4431-ba3c-ed92b3ce9a1d admin admin] The token cache is being invalidate because project 740697ca9b09496396cc8a471eb3451d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:16:05.550337 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e4ff14b4-6763-4431-ba3c-ed92b3ce9a1d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:05.551146 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e4ff14b4-6763-4431-ba3c-ed92b3ce9a1d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:05.552741 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1171/2354] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:16:05 2022] DELETE /identity/v3/projects/740697ca9b09496396cc8a471eb3451d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:16:05.558270 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f2990e-f597-40bf-a4de-b10b73a79452 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:05.559300 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f2990e-f597-40bf-a4de-b10b73a79452 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:05.559652 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f2990e-f597-40bf-a4de-b10b73a79452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:05.585214 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-d0f2990e-f597-40bf-a4de-b10b73a79452 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:16:05.587146 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-d0f2990e-f597-40bf-a4de-b10b73a79452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:05.655350 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1184/2355] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:16:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:16:05.662808 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21fdcf7-9152-426a-bc69-c6d20307a5ec None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:05.663169 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21fdcf7-9152-426a-bc69-c6d20307a5ec None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:05.663521 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21fdcf7-9152-426a-bc69-c6d20307a5ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:05.684121 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-e21fdcf7-9152-426a-bc69-c6d20307a5ec None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:16:05.685385 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e21fdcf7-9152-426a-bc69-c6d20307a5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:05.694292 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1172/2356] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:16:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:16:05.698499 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f2990e-f597-40bf-a4de-b10b73a79452 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:05.708969 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f2990e-f597-40bf-a4de-b10b73a79452 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:05.719944 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e99e32-bf32-4497-a975-00b48c6a9ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:05.720581 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e99e32-bf32-4497-a975-00b48c6a9ffb admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:05.720862 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e99e32-bf32-4497-a975-00b48c6a9ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:05.721135 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e99e32-bf32-4497-a975-00b48c6a9ffb admin admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:05.721799 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e99e32-bf32-4497-a975-00b48c6a9ffb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:16:05.722105 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e99e32-bf32-4497-a975-00b48c6a9ffb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:05.723785 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e99e32-bf32-4497-a975-00b48c6a9ffb admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:05.736816 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1185/2357] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:16:05.741132 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21fdcf7-9152-426a-bc69-c6d20307a5ec None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:05.750967 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21fdcf7-9152-426a-bc69-c6d20307a5ec None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:05.755810 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea8024c-ebb4-46b1-b86a-17c76f7c745e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:05.757045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8024c-ebb4-46b1-b86a-17c76f7c745e admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:05.757425 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8024c-ebb4-46b1-b86a-17c76f7c745e admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:05.757701 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea8024c-ebb4-46b1-b86a-17c76f7c745e admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:05.758391 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea8024c-ebb4-46b1-b86a-17c76f7c745e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:05.760317 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea8024c-ebb4-46b1-b86a-17c76f7c745e admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:05.789796 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1173/2358] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:16:05 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:16:05.794172 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e99e32-bf32-4497-a975-00b48c6a9ffb admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:05.804931 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e99e32-bf32-4497-a975-00b48c6a9ffb admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:05.864622 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1be582-47f6-4c2f-bfae-80fbd9d98fd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:05.865243 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1be582-47f6-4c2f-bfae-80fbd9d98fd9 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:05.865538 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1be582-47f6-4c2f-bfae-80fbd9d98fd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:05.865797 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1be582-47f6-4c2f-bfae-80fbd9d98fd9 admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:05.866469 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1be582-47f6-4c2f-bfae-80fbd9d98fd9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:05.868413 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1be582-47f6-4c2f-bfae-80fbd9d98fd9 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:05.892118 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1186/2359] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:16:05 2022] POST /identity/v3/users => generated 286 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:16:05.896744 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea8024c-ebb4-46b1-b86a-17c76f7c745e admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:05.908677 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea8024c-ebb4-46b1-b86a-17c76f7c745e admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:05.913585 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9121f793-0299-4ca9-aa65-0b8cc70d8255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:05.914197 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121f793-0299-4ca9-aa65-0b8cc70d8255 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:05.914508 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121f793-0299-4ca9-aa65-0b8cc70d8255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:05.914776 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9121f793-0299-4ca9-aa65-0b8cc70d8255 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:05.915451 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9121f793-0299-4ca9-aa65-0b8cc70d8255 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:05.917040 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9121f793-0299-4ca9-aa65-0b8cc70d8255 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:05.927773 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1174/2360] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:16:05.931975 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1be582-47f6-4c2f-bfae-80fbd9d98fd9 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:05.937469 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:16:05.944299 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1be582-47f6-4c2f-bfae-80fbd9d98fd9 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:05.950738 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e879f4b-76c9-4c5f-9c10-7309dd0d56fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:05.951350 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e879f4b-76c9-4c5f-9c10-7309dd0d56fd admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:05.951632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e879f4b-76c9-4c5f-9c10-7309dd0d56fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:05.951906 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e879f4b-76c9-4c5f-9c10-7309dd0d56fd admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:05.952559 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e879f4b-76c9-4c5f-9c10-7309dd0d56fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:05.954168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e879f4b-76c9-4c5f-9c10-7309dd0d56fd admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:05.966197 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1187/2361] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:16:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:16:05.970629 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9121f793-0299-4ca9-aa65-0b8cc70d8255 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:05.981192 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9121f793-0299-4ca9-aa65-0b8cc70d8255 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:05.985820 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e4f094-335b-4ce2-8ded-f66ef5a40be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:05.986659 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4f094-335b-4ce2-8ded-f66ef5a40be3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:05.986941 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4f094-335b-4ce2-8ded-f66ef5a40be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:05.987215 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e4f094-335b-4ce2-8ded-f66ef5a40be3 admin admin] PATH_INFO: `/v3/projects/73e4af79fa3f45e49e85944b7821c6cd/users/930223dd207e47608666443a435957aa/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:06.002513 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e4f094-335b-4ce2-8ded-f66ef5a40be3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73e4af79fa3f45e49e85944b7821c6cd, user_id=930223dd207e47608666443a435957aa, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:06.005173 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e4f094-335b-4ce2-8ded-f66ef5a40be3 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:06.022931 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1175/2362] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:16:05 2022] PUT /identity/v3/projects/73e4af79fa3f45e49e85944b7821c6cd/users/930223dd207e47608666443a435957aa/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:16:06.034416 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b9105-1e98-4678-99e2-5198cae7d35c None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:06.034713 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b9105-1e98-4678-99e2-5198cae7d35c None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:06.034986 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9b9105-1e98-4678-99e2-5198cae7d35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:06.063230 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-bc9b9105-1e98-4678-99e2-5198cae7d35c None None] MFA Rules not processed for user `930223dd207e47608666443a435957aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:16:06.064369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-bc9b9105-1e98-4678-99e2-5198cae7d35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:06.129894 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1188/2363] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:16:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:16:06.141502 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e4f094-335b-4ce2-8ded-f66ef5a40be3 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:06.143293 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-79e4f094-335b-4ce2-8ded-f66ef5a40be3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:06.200964 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e4f094-335b-4ce2-8ded-f66ef5a40be3 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:06.204591 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3cf9c8-a78f-48b3-a17d-11935dcced62 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:06.205194 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cf9c8-a78f-48b3-a17d-11935dcced62 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:06.205472 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cf9c8-a78f-48b3-a17d-11935dcced62 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:06.205746 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3cf9c8-a78f-48b3-a17d-11935dcced62 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:06.213943 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cf9c8-a78f-48b3-a17d-11935dcced62 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:06.215431 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3cf9c8-a78f-48b3-a17d-11935dcced62 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:06.224215 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1176/2364] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:16:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:16:06.246638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b9105-1e98-4678-99e2-5198cae7d35c None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:06.310487 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9b9105-1e98-4678-99e2-5198cae7d35c None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:06.315265 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ddd97-8ce9-4809-bf9c-67ba1e666d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:06.315828 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ddd97-8ce9-4809-bf9c-67ba1e666d4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:06.316102 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ddd97-8ce9-4809-bf9c-67ba1e666d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:06.316371 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ddd97-8ce9-4809-bf9c-67ba1e666d4c admin admin] PATH_INFO: `/v3/users/930223dd207e47608666443a435957aa` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:06.317481 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9ddd97-8ce9-4809-bf9c-67ba1e666d4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=930223dd207e47608666443a435957aa)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:06.320014 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9ddd97-8ce9-4809-bf9c-67ba1e666d4c admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:06.364693 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ab9ddd97-8ce9-4809-bf9c-67ba1e666d4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '930223dd207e47608666443a435957aa'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:06.376985 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ab9ddd97-8ce9-4809-bf9c-67ba1e666d4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '930223dd207e47608666443a435957aa'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:06.384405 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ab9ddd97-8ce9-4809-bf9c-67ba1e666d4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '930223dd207e47608666443a435957aa'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:06.391419 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ab9ddd97-8ce9-4809-bf9c-67ba1e666d4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '930223dd207e47608666443a435957aa'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:06.392056 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ab9ddd97-8ce9-4809-bf9c-67ba1e666d4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '930223dd207e47608666443a435957aa'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:06.394127 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:16:06.398222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1189/2365] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:16:06 2022] DELETE /identity/v3/users/930223dd207e47608666443a435957aa => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:16:06.402782 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ebd91c-f1f9-4c6d-b504-17d26a35e0a3 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:06.403331 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ebd91c-f1f9-4c6d-b504-17d26a35e0a3 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:06.403508 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ebd91c-f1f9-4c6d-b504-17d26a35e0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:06.411913 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:16:06.425318 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-c6ebd91c-f1f9-4c6d-b504-17d26a35e0a3 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:16:06.426578 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c6ebd91c-f1f9-4c6d-b504-17d26a35e0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:06.486830 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1177/2366] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:16:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:16:06.497456 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ddd97-8ce9-4809-bf9c-67ba1e666d4c admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:06.498897 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-ab9ddd97-8ce9-4809-bf9c-67ba1e666d4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:06.557525 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ddd97-8ce9-4809-bf9c-67ba1e666d4c admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:06.561243 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879fc2bd-fe69-44f0-a7e5-638df3bfb43f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:06.561848 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fc2bd-fe69-44f0-a7e5-638df3bfb43f service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:06.562124 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fc2bd-fe69-44f0-a7e5-638df3bfb43f service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:06.562428 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879fc2bd-fe69-44f0-a7e5-638df3bfb43f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:06.571880 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879fc2bd-fe69-44f0-a7e5-638df3bfb43f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:06.573299 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879fc2bd-fe69-44f0-a7e5-638df3bfb43f service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:06.583090 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1190/2367] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:16:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:16:06.857237 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ebd91c-f1f9-4c6d-b504-17d26a35e0a3 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:06.858705 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c6ebd91c-f1f9-4c6d-b504-17d26a35e0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:06.870472 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ebd91c-f1f9-4c6d-b504-17d26a35e0a3 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:06.876189 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a4941b-b61d-4c70-8625-55c002604b04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:06.876838 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4941b-b61d-4c70-8625-55c002604b04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:06.877196 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4941b-b61d-4c70-8625-55c002604b04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:06.877557 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a4941b-b61d-4c70-8625-55c002604b04 admin admin] PATH_INFO: `/v3/projects/73e4af79fa3f45e49e85944b7821c6cd` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:06.878675 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a4941b-b61d-4c70-8625-55c002604b04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73e4af79fa3f45e49e85944b7821c6cd)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:06.880681 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a4941b-b61d-4c70-8625-55c002604b04 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:06.927765 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f1a4941b-b61d-4c70-8625-55c002604b04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73e4af79fa3f45e49e85944b7821c6cd', 'request_id': 'req-f1a4941b-b61d-4c70-8625-55c002604b04'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:06.934724 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f1a4941b-b61d-4c70-8625-55c002604b04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73e4af79fa3f45e49e85944b7821c6cd', 'request_id': 'req-f1a4941b-b61d-4c70-8625-55c002604b04'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:06.940379 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f1a4941b-b61d-4c70-8625-55c002604b04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73e4af79fa3f45e49e85944b7821c6cd', 'request_id': 'req-f1a4941b-b61d-4c70-8625-55c002604b04'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:06.951861 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f1a4941b-b61d-4c70-8625-55c002604b04 admin admin] The token cache is being invalidate because project 73e4af79fa3f45e49e85944b7821c6cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:16:06.952149 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f1a4941b-b61d-4c70-8625-55c002604b04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:06.952788 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f1a4941b-b61d-4c70-8625-55c002604b04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:06.954278 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1178/2368] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:16:06 2022] DELETE /identity/v3/projects/73e4af79fa3f45e49e85944b7821c6cd => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:16:06.959189 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2cd9d0-b5a3-400b-9a46-37880ec3e07d None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:06.959978 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2cd9d0-b5a3-400b-9a46-37880ec3e07d None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:06.960258 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2cd9d0-b5a3-400b-9a46-37880ec3e07d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:06.980361 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-ea2cd9d0-b5a3-400b-9a46-37880ec3e07d None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:16:06.981571 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-ea2cd9d0-b5a3-400b-9a46-37880ec3e07d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:07.042114 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1191/2369] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:16:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:16:07.049675 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df1f3a-8b3a-4ec4-8872-938ae21c1aa5 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:07.049961 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df1f3a-8b3a-4ec4-8872-938ae21c1aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:07.050252 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df1f3a-8b3a-4ec4-8872-938ae21c1aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:07.069807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-65df1f3a-8b3a-4ec4-8872-938ae21c1aa5 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:16:07.071074 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-65df1f3a-8b3a-4ec4-8872-938ae21c1aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:07.079247 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1179/2370] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:16:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:16:07.083207 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2cd9d0-b5a3-400b-9a46-37880ec3e07d None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:07.093163 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2cd9d0-b5a3-400b-9a46-37880ec3e07d None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:07.097810 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd8868-99c5-45c2-98df-6e34bfc15693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:07.098442 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd8868-99c5-45c2-98df-6e34bfc15693 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:07.098720 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd8868-99c5-45c2-98df-6e34bfc15693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:07.098989 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd8868-99c5-45c2-98df-6e34bfc15693 admin admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:07.099680 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fd8868-99c5-45c2-98df-6e34bfc15693 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:16:07.100524 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fd8868-99c5-45c2-98df-6e34bfc15693 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:07.102107 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fd8868-99c5-45c2-98df-6e34bfc15693 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:07.113613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1192/2371] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:16:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 20:16:07.117905 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df1f3a-8b3a-4ec4-8872-938ae21c1aa5 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:07.127095 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df1f3a-8b3a-4ec4-8872-938ae21c1aa5 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:07.132370 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d993d7-a7e1-43be-b169-331e10e60856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:07.132965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d993d7-a7e1-43be-b169-331e10e60856 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:07.133247 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d993d7-a7e1-43be-b169-331e10e60856 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:07.133522 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d993d7-a7e1-43be-b169-331e10e60856 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:07.134164 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d993d7-a7e1-43be-b169-331e10e60856 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:07.136091 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d993d7-a7e1-43be-b169-331e10e60856 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:07.163036 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1180/2372] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:16:07 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:16:07.167096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd8868-99c5-45c2-98df-6e34bfc15693 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:07.177270 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd8868-99c5-45c2-98df-6e34bfc15693 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:07.233345 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da568c4-45ec-4b87-9989-c720d01ba7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:07.233949 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da568c4-45ec-4b87-9989-c720d01ba7f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:07.234231 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da568c4-45ec-4b87-9989-c720d01ba7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:07.234538 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da568c4-45ec-4b87-9989-c720d01ba7f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:07.235182 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da568c4-45ec-4b87-9989-c720d01ba7f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:07.237087 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da568c4-45ec-4b87-9989-c720d01ba7f0 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:07.257974 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1193/2373] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:16:07 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:16:07.261992 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d993d7-a7e1-43be-b169-331e10e60856 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:07.272877 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d993d7-a7e1-43be-b169-331e10e60856 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:07.277543 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33398d29-997a-44c3-925a-9f1b6e7222c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:07.278141 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33398d29-997a-44c3-925a-9f1b6e7222c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:07.278463 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33398d29-997a-44c3-925a-9f1b6e7222c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:07.278741 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33398d29-997a-44c3-925a-9f1b6e7222c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:07.279387 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33398d29-997a-44c3-925a-9f1b6e7222c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:07.280970 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33398d29-997a-44c3-925a-9f1b6e7222c8 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:07.291024 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1181/2374] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:16:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:16:07.294901 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da568c4-45ec-4b87-9989-c720d01ba7f0 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:07.304889 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da568c4-45ec-4b87-9989-c720d01ba7f0 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:07.309482 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae656ccc-f937-4eb0-a54d-43905722a964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:07.310076 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae656ccc-f937-4eb0-a54d-43905722a964 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:07.310387 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae656ccc-f937-4eb0-a54d-43905722a964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:07.310663 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae656ccc-f937-4eb0-a54d-43905722a964 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:07.311305 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae656ccc-f937-4eb0-a54d-43905722a964 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:07.312899 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae656ccc-f937-4eb0-a54d-43905722a964 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:07.323864 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1194/2375] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:16:07 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:16:07.327950 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33398d29-997a-44c3-925a-9f1b6e7222c8 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:07.338356 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33398d29-997a-44c3-925a-9f1b6e7222c8 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:07.343014 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b2cb2-aa9d-474e-bd17-94575aa756ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:07.343661 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b2cb2-aa9d-474e-bd17-94575aa756ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:07.343943 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b2cb2-aa9d-474e-bd17-94575aa756ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:07.344221 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65b2cb2-aa9d-474e-bd17-94575aa756ca admin admin] PATH_INFO: `/v3/projects/838cc64d58874cd1af4b7344db77ee64/users/dac36dabb9c6413b9726d93c47d752cc/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:07.358873 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b2cb2-aa9d-474e-bd17-94575aa756ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=838cc64d58874cd1af4b7344db77ee64, user_id=dac36dabb9c6413b9726d93c47d752cc, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:07.361533 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65b2cb2-aa9d-474e-bd17-94575aa756ca admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:07.374542 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1182/2376] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:16:07 2022] PUT /identity/v3/projects/838cc64d58874cd1af4b7344db77ee64/users/dac36dabb9c6413b9726d93c47d752cc/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:16:07.385164 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58a70b-fb6e-4e5b-b872-6524af4fe2ac None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:07.385499 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58a70b-fb6e-4e5b-b872-6524af4fe2ac None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:07.385782 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e58a70b-fb6e-4e5b-b872-6524af4fe2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:07.405632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:16:07.415244 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-0e58a70b-fb6e-4e5b-b872-6524af4fe2ac None None] MFA Rules not processed for user `dac36dabb9c6413b9726d93c47d752cc`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:16:07.416458 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-0e58a70b-fb6e-4e5b-b872-6524af4fe2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:07.479029 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1195/2377] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:16:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:16:07.489838 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b2cb2-aa9d-474e-bd17-94575aa756ca admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:07.491612 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f65b2cb2-aa9d-474e-bd17-94575aa756ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:07.519122 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:16:07.550105 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65b2cb2-aa9d-474e-bd17-94575aa756ca admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:07.554278 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e217cc-8e82-4169-bd77-30cebccb33c9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '021216f90d7f4ff6b6eb7a1d86399d99', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:07.554889 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e217cc-8e82-4169-bd77-30cebccb33c9 service glance] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:07.555168 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e217cc-8e82-4169-bd77-30cebccb33c9 service glance] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:07.555441 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e217cc-8e82-4169-bd77-30cebccb33c9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:07.563450 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e217cc-8e82-4169-bd77-30cebccb33c9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:07.564856 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e217cc-8e82-4169-bd77-30cebccb33c9 service glance] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:07.574334 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1183/2378] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:16:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 20:16:17.476094 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=28 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:16:17.570209 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=15 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:16:32.593638 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58a70b-fb6e-4e5b-b872-6524af4fe2ac None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:32.654871 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e58a70b-fb6e-4e5b-b872-6524af4fe2ac None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:32.659364 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf90792-05a9-4a79-935a-81e775f78a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:32.659984 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf90792-05a9-4a79-935a-81e775f78a64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:32.660255 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf90792-05a9-4a79-935a-81e775f78a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:32.660532 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf90792-05a9-4a79-935a-81e775f78a64 admin admin] PATH_INFO: `/v3/users/dac36dabb9c6413b9726d93c47d752cc` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:32.661636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf90792-05a9-4a79-935a-81e775f78a64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dac36dabb9c6413b9726d93c47d752cc)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:32.663564 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf90792-05a9-4a79-935a-81e775f78a64 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:32.709132 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-bbf90792-05a9-4a79-935a-81e775f78a64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dac36dabb9c6413b9726d93c47d752cc'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:32.723183 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-bbf90792-05a9-4a79-935a-81e775f78a64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dac36dabb9c6413b9726d93c47d752cc'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:32.731577 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-bbf90792-05a9-4a79-935a-81e775f78a64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dac36dabb9c6413b9726d93c47d752cc'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:32.740703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-bbf90792-05a9-4a79-935a-81e775f78a64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dac36dabb9c6413b9726d93c47d752cc'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:32.741291 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-bbf90792-05a9-4a79-935a-81e775f78a64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dac36dabb9c6413b9726d93c47d752cc'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:32.752610 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1196/2379] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:16:32 2022] DELETE /identity/v3/users/dac36dabb9c6413b9726d93c47d752cc => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:16:32.754312 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de244d6-2921-4b18-a3b7-3b438be0d99b None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:32.754768 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de244d6-2921-4b18-a3b7-3b438be0d99b None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:32.755060 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de244d6-2921-4b18-a3b7-3b438be0d99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:32.774761 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-1de244d6-2921-4b18-a3b7-3b438be0d99b None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:16:32.776017 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-1de244d6-2921-4b18-a3b7-3b438be0d99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:32.838708 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1184/2380] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:16:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:16:32.849437 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf90792-05a9-4a79-935a-81e775f78a64 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:32.851569 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-bbf90792-05a9-4a79-935a-81e775f78a64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:32.913823 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf90792-05a9-4a79-935a-81e775f78a64 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:32.918142 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec38526-0af1-4e85-b305-2013c143ea0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:32.918972 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec38526-0af1-4e85-b305-2013c143ea0e service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:32.919255 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec38526-0af1-4e85-b305-2013c143ea0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:32.919531 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec38526-0af1-4e85-b305-2013c143ea0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:32.932333 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec38526-0af1-4e85-b305-2013c143ea0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:32.933775 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec38526-0af1-4e85-b305-2013c143ea0e service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:32.944071 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1197/2381] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:16:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:16:33.452001 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de244d6-2921-4b18-a3b7-3b438be0d99b None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:33.454428 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-1de244d6-2921-4b18-a3b7-3b438be0d99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:33.489471 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de244d6-2921-4b18-a3b7-3b438be0d99b None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:33.505874 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7007edb-5154-44f2-8708-20f04acf51e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:33.506557 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7007edb-5154-44f2-8708-20f04acf51e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:33.507099 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7007edb-5154-44f2-8708-20f04acf51e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:33.507382 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7007edb-5154-44f2-8708-20f04acf51e1 admin admin] PATH_INFO: `/v3/projects/838cc64d58874cd1af4b7344db77ee64` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:33.509133 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7007edb-5154-44f2-8708-20f04acf51e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=838cc64d58874cd1af4b7344db77ee64)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:33.511340 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7007edb-5154-44f2-8708-20f04acf51e1 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:33.559732 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d7007edb-5154-44f2-8708-20f04acf51e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '838cc64d58874cd1af4b7344db77ee64', 'request_id': 'req-d7007edb-5154-44f2-8708-20f04acf51e1'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:33.567997 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d7007edb-5154-44f2-8708-20f04acf51e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '838cc64d58874cd1af4b7344db77ee64', 'request_id': 'req-d7007edb-5154-44f2-8708-20f04acf51e1'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:33.574023 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d7007edb-5154-44f2-8708-20f04acf51e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '838cc64d58874cd1af4b7344db77ee64', 'request_id': 'req-d7007edb-5154-44f2-8708-20f04acf51e1'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:33.587568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d7007edb-5154-44f2-8708-20f04acf51e1 admin admin] The token cache is being invalidate because project 838cc64d58874cd1af4b7344db77ee64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:16:33.587859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d7007edb-5154-44f2-8708-20f04acf51e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:33.588511 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-d7007edb-5154-44f2-8708-20f04acf51e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:33.590028 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1185/2382] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:16:33 2022] DELETE /identity/v3/projects/838cc64d58874cd1af4b7344db77ee64 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:16:33.595075 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1c253-b72f-467f-b79e-0d78fb4d7ed8 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:33.595929 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1c253-b72f-467f-b79e-0d78fb4d7ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:33.596255 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1c253-b72f-467f-b79e-0d78fb4d7ed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:33.617180 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-75b1c253-b72f-467f-b79e-0d78fb4d7ed8 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:16:33.618992 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-75b1c253-b72f-467f-b79e-0d78fb4d7ed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:33.638657 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:16:33.688956 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7007edb-5154-44f2-8708-20f04acf51e1 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:33.690621 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d7007edb-5154-44f2-8708-20f04acf51e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:33.702365 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1198/2383] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:16:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:16:33.772542 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e904d-11dc-47c0-ad71-d30413432f03 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:33.772834 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e904d-11dc-47c0-ad71-d30413432f03 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:33.773122 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e904d-11dc-47c0-ad71-d30413432f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:33.798020 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-bf1e904d-11dc-47c0-ad71-d30413432f03 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:16:33.799773 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-bf1e904d-11dc-47c0-ad71-d30413432f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:33.818477 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1199/2384] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:16:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:16:33.820605 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7007edb-5154-44f2-8708-20f04acf51e1 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:33.826088 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e904d-11dc-47c0-ad71-d30413432f03 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:33.828288 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1319746-e9d3-433f-a57f-4b84b31aed88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:33.828928 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1319746-e9d3-433f-a57f-4b84b31aed88 service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:33.829210 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1319746-e9d3-433f-a57f-4b84b31aed88 service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:33.829491 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1319746-e9d3-433f-a57f-4b84b31aed88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:33.831610 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c1319746-e9d3-433f-a57f-4b84b31aed88 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:33.847486 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1319746-e9d3-433f-a57f-4b84b31aed88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:33.849101 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1319746-e9d3-433f-a57f-4b84b31aed88 service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:33.849858 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e904d-11dc-47c0-ad71-d30413432f03 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:33.863098 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bea8c9-2270-42ba-84f5-f798a240f7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:33.865025 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1186/2385] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:16:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 20:16:33.865230 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bea8c9-2270-42ba-84f5-f798a240f7f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:33.865554 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bea8c9-2270-42ba-84f5-f798a240f7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:33.865832 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bea8c9-2270-42ba-84f5-f798a240f7f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:33.868044 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bea8c9-2270-42ba-84f5-f798a240f7f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:16:33.868432 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bea8c9-2270-42ba-84f5-f798a240f7f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:33.870156 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bea8c9-2270-42ba-84f5-f798a240f7f0 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:33.897550 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1200/2386] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:16:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:16:33.902127 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1319746-e9d3-433f-a57f-4b84b31aed88 service nova] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:33.921421 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1319746-e9d3-433f-a57f-4b84b31aed88 service nova] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:33.928755 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2349335-83c8-4275-8ba0-e991820538da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:33.929475 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2349335-83c8-4275-8ba0-e991820538da admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:33.929957 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2349335-83c8-4275-8ba0-e991820538da admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:33.930576 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2349335-83c8-4275-8ba0-e991820538da admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:33.931245 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2349335-83c8-4275-8ba0-e991820538da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:33.934519 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2349335-83c8-4275-8ba0-e991820538da admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:33.941310 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:16:33.989112 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1187/2387] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:16:33 2022] POST /identity/v3/projects => generated 385 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:16:33.992892 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bea8c9-2270-42ba-84f5-f798a240f7f0 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:34.013075 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bea8c9-2270-42ba-84f5-f798a240f7f0 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:34.072162 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88973161-aa18-43df-80a7-9db0672e0ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:34.072776 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88973161-aa18-43df-80a7-9db0672e0ce7 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:34.073053 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88973161-aa18-43df-80a7-9db0672e0ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:34.073443 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88973161-aa18-43df-80a7-9db0672e0ce7 admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:34.074089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88973161-aa18-43df-80a7-9db0672e0ce7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:34.076288 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88973161-aa18-43df-80a7-9db0672e0ce7 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:34.098442 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1201/2388] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:16:33 2022] POST /identity/v3/users => generated 293 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:16:34.102403 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2349335-83c8-4275-8ba0-e991820538da admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:34.112848 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2349335-83c8-4275-8ba0-e991820538da admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:34.117727 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889bcced-5c33-4001-b553-efb85d6b5b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:34.118565 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bcced-5c33-4001-b553-efb85d6b5b4f admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:34.118847 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bcced-5c33-4001-b553-efb85d6b5b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:34.119121 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889bcced-5c33-4001-b553-efb85d6b5b4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:34.119764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889bcced-5c33-4001-b553-efb85d6b5b4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:34.121406 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889bcced-5c33-4001-b553-efb85d6b5b4f admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:34.131600 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1188/2389] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 20:16:34.136431 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88973161-aa18-43df-80a7-9db0672e0ce7 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:34.146480 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88973161-aa18-43df-80a7-9db0672e0ce7 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:34.151702 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d182465e-e85f-4097-b869-62643a157aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:34.152304 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182465e-e85f-4097-b869-62643a157aa9 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:34.152598 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182465e-e85f-4097-b869-62643a157aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:34.152857 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d182465e-e85f-4097-b869-62643a157aa9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:34.153502 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d182465e-e85f-4097-b869-62643a157aa9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:34.155132 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d182465e-e85f-4097-b869-62643a157aa9 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:34.167232 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1202/2390] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:16:34.171259 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889bcced-5c33-4001-b553-efb85d6b5b4f admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:34.181173 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889bcced-5c33-4001-b553-efb85d6b5b4f admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:34.185825 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe55c3-c745-4a42-a80e-4135b316b251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:34.186506 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe55c3-c745-4a42-a80e-4135b316b251 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:34.186790 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe55c3-c745-4a42-a80e-4135b316b251 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:34.187061 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbe55c3-c745-4a42-a80e-4135b316b251 admin admin] PATH_INFO: `/v3/projects/e2ea5ffdf70947559bea8d76884c787e/users/41681782cea7468cbcb7d0441acc0907/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:34.202319 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbe55c3-c745-4a42-a80e-4135b316b251 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2ea5ffdf70947559bea8d76884c787e, user_id=41681782cea7468cbcb7d0441acc0907, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:34.204937 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbe55c3-c745-4a42-a80e-4135b316b251 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:34.219217 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1189/2391] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:16:34 2022] PUT /identity/v3/projects/e2ea5ffdf70947559bea8d76884c787e/users/41681782cea7468cbcb7d0441acc0907/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:16:34.229352 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59dc61b-f7f3-4508-bf37-20f5a3fba693 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:34.229695 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59dc61b-f7f3-4508-bf37-20f5a3fba693 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:34.230037 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59dc61b-f7f3-4508-bf37-20f5a3fba693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:34.257712 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-e59dc61b-f7f3-4508-bf37-20f5a3fba693 None None] MFA Rules not processed for user `41681782cea7468cbcb7d0441acc0907`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:16:34.259591 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-e59dc61b-f7f3-4508-bf37-20f5a3fba693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:34.341570 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe55c3-c745-4a42-a80e-4135b316b251 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:34.343567 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1203/2392] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:16:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:16:34.424344 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbe55c3-c745-4a42-a80e-4135b316b251 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:34.429186 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c81d6c-0615-462f-8a63-c562d609dd9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:34.429894 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c81d6c-0615-462f-8a63-c562d609dd9f admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:34.430306 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c81d6c-0615-462f-8a63-c562d609dd9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:34.430644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c81d6c-0615-462f-8a63-c562d609dd9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:34.431354 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c81d6c-0615-462f-8a63-c562d609dd9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:34.433310 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c81d6c-0615-462f-8a63-c562d609dd9f admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:34.463904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1190/2393] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:16:34 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:16:34.467991 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59dc61b-f7f3-4508-bf37-20f5a3fba693 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:34.479657 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59dc61b-f7f3-4508-bf37-20f5a3fba693 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:34.488038 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:16:34.548907 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e4ea74-8b28-4a37-b6f1-9db407097e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:34.549523 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4ea74-8b28-4a37-b6f1-9db407097e27 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:34.549803 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4ea74-8b28-4a37-b6f1-9db407097e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:34.550076 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4ea74-8b28-4a37-b6f1-9db407097e27 admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:34.550984 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e4ea74-8b28-4a37-b6f1-9db407097e27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:34.555978 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e4ea74-8b28-4a37-b6f1-9db407097e27 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:34.578486 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1204/2394] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:16:34 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:16:34.582682 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c81d6c-0615-462f-8a63-c562d609dd9f admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:34.593604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c81d6c-0615-462f-8a63-c562d609dd9f admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:34.599033 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea300ae-5b28-4ac2-ac79-6c782fe7fc7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:34.599863 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea300ae-5b28-4ac2-ac79-6c782fe7fc7c admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:34.600214 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea300ae-5b28-4ac2-ac79-6c782fe7fc7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:34.600584 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea300ae-5b28-4ac2-ac79-6c782fe7fc7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:34.601324 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea300ae-5b28-4ac2-ac79-6c782fe7fc7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:34.603131 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea300ae-5b28-4ac2-ac79-6c782fe7fc7c admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:34.615371 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1191/2395] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:16:34.619914 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e4ea74-8b28-4a37-b6f1-9db407097e27 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:34.630278 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e4ea74-8b28-4a37-b6f1-9db407097e27 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:34.636233 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f75f19f-6186-4c0f-8631-d0360d99c59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:34.637173 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f75f19f-6186-4c0f-8631-d0360d99c59c admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:34.637710 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f75f19f-6186-4c0f-8631-d0360d99c59c admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:34.638167 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f75f19f-6186-4c0f-8631-d0360d99c59c admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:34.639164 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f75f19f-6186-4c0f-8631-d0360d99c59c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:34.640678 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f75f19f-6186-4c0f-8631-d0360d99c59c admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:34.651988 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1205/2396] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:16:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:16:34.656456 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea300ae-5b28-4ac2-ac79-6c782fe7fc7c admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:34.667134 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea300ae-5b28-4ac2-ac79-6c782fe7fc7c admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:34.672872 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b9233-c170-4977-b32a-f7f632cbe39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:34.673543 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b9233-c170-4977-b32a-f7f632cbe39a admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:34.673820 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b9233-c170-4977-b32a-f7f632cbe39a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:34.674095 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b9233-c170-4977-b32a-f7f632cbe39a admin admin] PATH_INFO: `/v3/projects/8edcd5122ff74da8921903566ea0d97d/users/abd6d34c6b95450187abd773e0e532ea/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:34.694173 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56b9233-c170-4977-b32a-f7f632cbe39a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8edcd5122ff74da8921903566ea0d97d, user_id=abd6d34c6b95450187abd773e0e532ea, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:34.697450 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56b9233-c170-4977-b32a-f7f632cbe39a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:34.712076 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1192/2397] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:16:34 2022] PUT /identity/v3/projects/8edcd5122ff74da8921903566ea0d97d/users/abd6d34c6b95450187abd773e0e532ea/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:16:34.722718 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c0a34-2932-4992-8a64-09fd58f49bb9 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:34.723018 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c0a34-2932-4992-8a64-09fd58f49bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:34.723357 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c0a34-2932-4992-8a64-09fd58f49bb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:34.761853 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-ce1c0a34-2932-4992-8a64-09fd58f49bb9 None None] MFA Rules not processed for user `abd6d34c6b95450187abd773e0e532ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:16:34.763110 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-ce1c0a34-2932-4992-8a64-09fd58f49bb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:34.849952 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1206/2398] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:16:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:16:34.854651 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b9233-c170-4977-b32a-f7f632cbe39a admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:34.923175 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b9233-c170-4977-b32a-f7f632cbe39a admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:34.927824 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9caff1-7a5e-4575-9f92-8208bf062656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:34.928428 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9caff1-7a5e-4575-9f92-8208bf062656 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:34.928704 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9caff1-7a5e-4575-9f92-8208bf062656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:34.928974 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9caff1-7a5e-4575-9f92-8208bf062656 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:34.929619 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9caff1-7a5e-4575-9f92-8208bf062656 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:34.932188 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9caff1-7a5e-4575-9f92-8208bf062656 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:34.937732 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:16:34.960148 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1193/2399] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:16:34 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:16:34.964423 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c0a34-2932-4992-8a64-09fd58f49bb9 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:34.975092 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c0a34-2932-4992-8a64-09fd58f49bb9 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:35.036147 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256860c0-0c00-445d-92c7-1fdbf3b6bd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:35.036802 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256860c0-0c00-445d-92c7-1fdbf3b6bd5f admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:35.037083 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256860c0-0c00-445d-92c7-1fdbf3b6bd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:35.037352 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256860c0-0c00-445d-92c7-1fdbf3b6bd5f admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:35.037998 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256860c0-0c00-445d-92c7-1fdbf3b6bd5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:35.040554 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256860c0-0c00-445d-92c7-1fdbf3b6bd5f admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:35.064366 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1207/2400] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:16:34 2022] POST /identity/v3/users => generated 298 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:16:35.068593 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9caff1-7a5e-4575-9f92-8208bf062656 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:35.079038 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9caff1-7a5e-4575-9f92-8208bf062656 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:35.083588 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e2dc12-74d9-460a-9dc0-5abfc9d59299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:35.084187 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e2dc12-74d9-460a-9dc0-5abfc9d59299 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:35.084462 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e2dc12-74d9-460a-9dc0-5abfc9d59299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:35.084732 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e2dc12-74d9-460a-9dc0-5abfc9d59299 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:35.085393 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e2dc12-74d9-460a-9dc0-5abfc9d59299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:35.087027 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e2dc12-74d9-460a-9dc0-5abfc9d59299 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:35.098795 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1194/2401] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:16:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:16:35.103173 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256860c0-0c00-445d-92c7-1fdbf3b6bd5f admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:35.114201 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256860c0-0c00-445d-92c7-1fdbf3b6bd5f admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:35.120182 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1382f-5d60-45ce-9b37-f5db04c3b6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:35.120771 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1382f-5d60-45ce-9b37-f5db04c3b6e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:35.121066 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1382f-5d60-45ce-9b37-f5db04c3b6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:35.121466 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c1382f-5d60-45ce-9b37-f5db04c3b6e7 admin admin] PATH_INFO: `/v3/projects/0473a97c404c46138b9f8766e5a29c02/users/f439beff4af44769ae6bcd4145c9899d/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:35.125822 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:16:35.139494 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c1382f-5d60-45ce-9b37-f5db04c3b6e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0473a97c404c46138b9f8766e5a29c02, user_id=f439beff4af44769ae6bcd4145c9899d, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:35.142370 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c1382f-5d60-45ce-9b37-f5db04c3b6e7 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:35.156117 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1208/2402] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:16:35 2022] PUT /identity/v3/projects/0473a97c404c46138b9f8766e5a29c02/users/f439beff4af44769ae6bcd4145c9899d/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:16:35.160039 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e2dc12-74d9-460a-9dc0-5abfc9d59299 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:35.170044 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e2dc12-74d9-460a-9dc0-5abfc9d59299 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:35.243742 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea51aea0-3f4c-4a41-be68-09878dbad970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:35.244548 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51aea0-3f4c-4a41-be68-09878dbad970 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:35.244980 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51aea0-3f4c-4a41-be68-09878dbad970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:35.245750 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea51aea0-3f4c-4a41-be68-09878dbad970 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:35.246071 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea51aea0-3f4c-4a41-be68-09878dbad970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:35.247702 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea51aea0-3f4c-4a41-be68-09878dbad970 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:35.258984 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1195/2403] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:16:35 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 20:16:35.263078 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1382f-5d60-45ce-9b37-f5db04c3b6e7 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:35.273405 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c1382f-5d60-45ce-9b37-f5db04c3b6e7 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:35.278082 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8b76f-3714-43aa-9aa1-4e06276f945d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:35.278841 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8b76f-3714-43aa-9aa1-4e06276f945d admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:35.279122 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8b76f-3714-43aa-9aa1-4e06276f945d admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:35.279398 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8b76f-3714-43aa-9aa1-4e06276f945d admin admin] PATH_INFO: `/v3/projects/0473a97c404c46138b9f8766e5a29c02/users/f439beff4af44769ae6bcd4145c9899d/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:35.281316 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b8b76f-3714-43aa-9aa1-4e06276f945d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0473a97c404c46138b9f8766e5a29c02, user_id=f439beff4af44769ae6bcd4145c9899d, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:35.284007 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b8b76f-3714-43aa-9aa1-4e06276f945d admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:35.299599 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1209/2404] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:16:35 2022] PUT /identity/v3/projects/0473a97c404c46138b9f8766e5a29c02/users/f439beff4af44769ae6bcd4145c9899d/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:16:35.310914 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf41a1-e9b8-4458-8392-6b189cb19283 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:35.311209 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf41a1-e9b8-4458-8392-6b189cb19283 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:35.311486 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdf41a1-e9b8-4458-8392-6b189cb19283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:35.340363 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-dbdf41a1-e9b8-4458-8392-6b189cb19283 None None] MFA Rules not processed for user `f439beff4af44769ae6bcd4145c9899d`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:16:35.341585 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-dbdf41a1-e9b8-4458-8392-6b189cb19283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:35.412764 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1196/2405] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:16:35 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:16:35.424359 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8b76f-3714-43aa-9aa1-4e06276f945d admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:35.426026 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-18b8b76f-3714-43aa-9aa1-4e06276f945d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:35.488482 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b8b76f-3714-43aa-9aa1-4e06276f945d admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:35.493252 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526664ba-175b-4b42-9252-98e48e678cc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:35.493877 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526664ba-175b-4b42-9252-98e48e678cc6 service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:35.494160 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526664ba-175b-4b42-9252-98e48e678cc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:35.494458 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526664ba-175b-4b42-9252-98e48e678cc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:35.558579 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526664ba-175b-4b42-9252-98e48e678cc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:35.560169 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526664ba-175b-4b42-9252-98e48e678cc6 service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:35.570345 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1210/2406] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:16:35 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:16:38.114840 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf41a1-e9b8-4458-8392-6b189cb19283 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:38.124196 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdf41a1-e9b8-4458-8392-6b189cb19283 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:38.127714 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5bcf7-d04d-4891-8e5f-9984bd660e0d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:38.128303 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5bcf7-d04d-4891-8e5f-9984bd660e0d service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:38.128579 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5bcf7-d04d-4891-8e5f-9984bd660e0d service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:38.128847 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5bcf7-d04d-4891-8e5f-9984bd660e0d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:38.182429 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:16:38.184606 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba5bcf7-d04d-4891-8e5f-9984bd660e0d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:38.186494 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba5bcf7-d04d-4891-8e5f-9984bd660e0d service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:38.197003 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1197/2407] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:16:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:16:38.705968 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526664ba-175b-4b42-9252-98e48e678cc6 service cinder] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:38.714737 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526664ba-175b-4b42-9252-98e48e678cc6 service cinder] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:38.718331 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3158ca3-4ea6-4944-9b86-1020e140b381 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:38.718927 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3158ca3-4ea6-4944-9b86-1020e140b381 service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:38.719205 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3158ca3-4ea6-4944-9b86-1020e140b381 service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:38.719477 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3158ca3-4ea6-4944-9b86-1020e140b381 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:38.728767 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3158ca3-4ea6-4944-9b86-1020e140b381 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:38.730171 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3158ca3-4ea6-4944-9b86-1020e140b381 service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:38.739504 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1211/2408] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:16:38 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:16:42.601386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5bcf7-d04d-4891-8e5f-9984bd660e0d service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:42.662144 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5bcf7-d04d-4891-8e5f-9984bd660e0d service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:42.666671 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7795b070-98c9-4faf-a23c-ed2351b2678c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:42.667304 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795b070-98c9-4faf-a23c-ed2351b2678c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:42.667568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795b070-98c9-4faf-a23c-ed2351b2678c admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:42.667841 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7795b070-98c9-4faf-a23c-ed2351b2678c admin admin] PATH_INFO: `/v3/users/41681782cea7468cbcb7d0441acc0907` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:42.669093 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7795b070-98c9-4faf-a23c-ed2351b2678c admin admin] RBAC: Authorizing `identity:delete_user(user_id=41681782cea7468cbcb7d0441acc0907)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:42.671048 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7795b070-98c9-4faf-a23c-ed2351b2678c admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:42.716060 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-7795b070-98c9-4faf-a23c-ed2351b2678c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41681782cea7468cbcb7d0441acc0907'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:42.727898 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-7795b070-98c9-4faf-a23c-ed2351b2678c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41681782cea7468cbcb7d0441acc0907'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:42.734997 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-7795b070-98c9-4faf-a23c-ed2351b2678c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41681782cea7468cbcb7d0441acc0907'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:42.735608 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-7795b070-98c9-4faf-a23c-ed2351b2678c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41681782cea7468cbcb7d0441acc0907'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:42.742975 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-7795b070-98c9-4faf-a23c-ed2351b2678c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41681782cea7468cbcb7d0441acc0907'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:42.748890 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1198/2409] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:16:42 2022] DELETE /identity/v3/users/41681782cea7468cbcb7d0441acc0907 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:16:42.753433 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d61e33-42da-41a8-88b9-acd7127a0c9f None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:42.753897 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d61e33-42da-41a8-88b9-acd7127a0c9f None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:42.753994 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d61e33-42da-41a8-88b9-acd7127a0c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:42.773460 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-11d61e33-42da-41a8-88b9-acd7127a0c9f None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:16:42.775029 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-11d61e33-42da-41a8-88b9-acd7127a0c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:42.833069 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1212/2410] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:16:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:16:42.842711 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7795b070-98c9-4faf-a23c-ed2351b2678c admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:42.844129 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7795b070-98c9-4faf-a23c-ed2351b2678c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:42.901021 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7795b070-98c9-4faf-a23c-ed2351b2678c admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:42.904717 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e3dcc-2a91-4cc4-86f7-393f9f7608b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:42.905316 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e3dcc-2a91-4cc4-86f7-393f9f7608b4 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:42.905593 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e3dcc-2a91-4cc4-86f7-393f9f7608b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:42.905867 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42e3dcc-2a91-4cc4-86f7-393f9f7608b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:42.914682 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42e3dcc-2a91-4cc4-86f7-393f9f7608b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:42.916089 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42e3dcc-2a91-4cc4-86f7-393f9f7608b4 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:42.925227 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1199/2411] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:16:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:16:43.188703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d61e33-42da-41a8-88b9-acd7127a0c9f None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:43.190089 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-11d61e33-42da-41a8-88b9-acd7127a0c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:43.201877 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d61e33-42da-41a8-88b9-acd7127a0c9f None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:43.206859 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57a41d-721d-403d-85e6-266709fd73bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:43.207505 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57a41d-721d-403d-85e6-266709fd73bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:43.207891 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57a41d-721d-403d-85e6-266709fd73bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:43.208172 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f57a41d-721d-403d-85e6-266709fd73bc admin admin] PATH_INFO: `/v3/projects/e2ea5ffdf70947559bea8d76884c787e` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:43.209340 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57a41d-721d-403d-85e6-266709fd73bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2ea5ffdf70947559bea8d76884c787e)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:43.211318 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f57a41d-721d-403d-85e6-266709fd73bc admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:43.220265 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:16:43.270310 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-2f57a41d-721d-403d-85e6-266709fd73bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2ea5ffdf70947559bea8d76884c787e', 'request_id': 'req-2f57a41d-721d-403d-85e6-266709fd73bc'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:43.279689 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-2f57a41d-721d-403d-85e6-266709fd73bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2ea5ffdf70947559bea8d76884c787e', 'request_id': 'req-2f57a41d-721d-403d-85e6-266709fd73bc'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:43.285997 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-2f57a41d-721d-403d-85e6-266709fd73bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2ea5ffdf70947559bea8d76884c787e', 'request_id': 'req-2f57a41d-721d-403d-85e6-266709fd73bc'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:43.301923 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-2f57a41d-721d-403d-85e6-266709fd73bc admin admin] The token cache is being invalidate because project e2ea5ffdf70947559bea8d76884c787e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:16:43.302453 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-2f57a41d-721d-403d-85e6-266709fd73bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:43.303219 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-2f57a41d-721d-403d-85e6-266709fd73bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:43.305759 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1213/2412] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:16:43 2022] DELETE /identity/v3/projects/e2ea5ffdf70947559bea8d76884c787e => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:16:43.309558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e3dcc-2a91-4cc4-86f7-393f9f7608b4 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:43.321254 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42e3dcc-2a91-4cc4-86f7-393f9f7608b4 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:43.335532 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:16:43.412838 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7cdc66-7efc-4be2-a940-acbc998c6688 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:43.413494 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7cdc66-7efc-4be2-a940-acbc998c6688 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:43.413808 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7cdc66-7efc-4be2-a940-acbc998c6688 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:43.414080 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7cdc66-7efc-4be2-a940-acbc998c6688 admin admin] PATH_INFO: `/v3/users/abd6d34c6b95450187abd773e0e532ea` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:43.418299 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7cdc66-7efc-4be2-a940-acbc998c6688 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abd6d34c6b95450187abd773e0e532ea)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:43.418549 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7cdc66-7efc-4be2-a940-acbc998c6688 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:43.472918 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-cd7cdc66-7efc-4be2-a940-acbc998c6688 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abd6d34c6b95450187abd773e0e532ea'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:43.485911 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-cd7cdc66-7efc-4be2-a940-acbc998c6688 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abd6d34c6b95450187abd773e0e532ea'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:43.495288 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-cd7cdc66-7efc-4be2-a940-acbc998c6688 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abd6d34c6b95450187abd773e0e532ea'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:43.495940 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-cd7cdc66-7efc-4be2-a940-acbc998c6688 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abd6d34c6b95450187abd773e0e532ea'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:43.503454 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-cd7cdc66-7efc-4be2-a940-acbc998c6688 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abd6d34c6b95450187abd773e0e532ea'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:43.510933 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1200/2413] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:16:43 2022] DELETE /identity/v3/users/abd6d34c6b95450187abd773e0e532ea => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:16:43.786704 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57a41d-721d-403d-85e6-266709fd73bc admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:43.788263 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-2f57a41d-721d-403d-85e6-266709fd73bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:43.802582 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f57a41d-721d-403d-85e6-266709fd73bc admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:43.867855 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b312e14-03ee-402c-9a57-9d8d4507bce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:43.868501 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b312e14-03ee-402c-9a57-9d8d4507bce7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:43.868783 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b312e14-03ee-402c-9a57-9d8d4507bce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:43.869065 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b312e14-03ee-402c-9a57-9d8d4507bce7 admin admin] PATH_INFO: `/v3/projects/8edcd5122ff74da8921903566ea0d97d` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:43.870190 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b312e14-03ee-402c-9a57-9d8d4507bce7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8edcd5122ff74da8921903566ea0d97d)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:43.872139 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b312e14-03ee-402c-9a57-9d8d4507bce7 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:43.921884 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-8b312e14-03ee-402c-9a57-9d8d4507bce7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8edcd5122ff74da8921903566ea0d97d', 'request_id': 'req-8b312e14-03ee-402c-9a57-9d8d4507bce7'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:43.929664 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-8b312e14-03ee-402c-9a57-9d8d4507bce7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8edcd5122ff74da8921903566ea0d97d', 'request_id': 'req-8b312e14-03ee-402c-9a57-9d8d4507bce7'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:43.936710 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-8b312e14-03ee-402c-9a57-9d8d4507bce7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8edcd5122ff74da8921903566ea0d97d', 'request_id': 'req-8b312e14-03ee-402c-9a57-9d8d4507bce7'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:43.948933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-8b312e14-03ee-402c-9a57-9d8d4507bce7 admin admin] The token cache is being invalidate because project 8edcd5122ff74da8921903566ea0d97d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:16:43.949230 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-8b312e14-03ee-402c-9a57-9d8d4507bce7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:43.949904 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-8b312e14-03ee-402c-9a57-9d8d4507bce7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:43.951783 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1214/2414] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:16:43 2022] DELETE /identity/v3/projects/8edcd5122ff74da8921903566ea0d97d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:16:43.955780 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7cdc66-7efc-4be2-a940-acbc998c6688 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:43.957316 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-cd7cdc66-7efc-4be2-a940-acbc998c6688 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:43.970658 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7cdc66-7efc-4be2-a940-acbc998c6688 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:44.031691 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353fef4b-af8c-4f17-98f9-6b9f708d086e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:44.032330 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fef4b-af8c-4f17-98f9-6b9f708d086e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:44.032609 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fef4b-af8c-4f17-98f9-6b9f708d086e admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:44.032888 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353fef4b-af8c-4f17-98f9-6b9f708d086e admin admin] PATH_INFO: `/v3/users/f439beff4af44769ae6bcd4145c9899d` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:44.034003 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353fef4b-af8c-4f17-98f9-6b9f708d086e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f439beff4af44769ae6bcd4145c9899d)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:44.035980 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353fef4b-af8c-4f17-98f9-6b9f708d086e admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:44.077798 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:16:44.084855 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-353fef4b-af8c-4f17-98f9-6b9f708d086e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f439beff4af44769ae6bcd4145c9899d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:44.098018 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-353fef4b-af8c-4f17-98f9-6b9f708d086e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f439beff4af44769ae6bcd4145c9899d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:44.105568 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-353fef4b-af8c-4f17-98f9-6b9f708d086e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f439beff4af44769ae6bcd4145c9899d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:44.106205 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-353fef4b-af8c-4f17-98f9-6b9f708d086e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f439beff4af44769ae6bcd4145c9899d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:44.113526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-353fef4b-af8c-4f17-98f9-6b9f708d086e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f439beff4af44769ae6bcd4145c9899d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:44.119819 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1201/2415] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:16:43 2022] DELETE /identity/v3/users/f439beff4af44769ae6bcd4145c9899d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:16:44.398931 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b312e14-03ee-402c-9a57-9d8d4507bce7 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:44.400397 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-8b312e14-03ee-402c-9a57-9d8d4507bce7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:44.414685 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b312e14-03ee-402c-9a57-9d8d4507bce7 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:44.455968 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:16:44.491717 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc8141c-e09c-424d-a1f0-77a5d406ddca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:44.493037 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8141c-e09c-424d-a1f0-77a5d406ddca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:44.495769 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8141c-e09c-424d-a1f0-77a5d406ddca admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:44.496171 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8141c-e09c-424d-a1f0-77a5d406ddca admin admin] PATH_INFO: `/v3/projects/0473a97c404c46138b9f8766e5a29c02` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:44.497774 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc8141c-e09c-424d-a1f0-77a5d406ddca admin admin] RBAC: Authorizing `identity:delete_project(project_id=0473a97c404c46138b9f8766e5a29c02)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:44.499847 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc8141c-e09c-424d-a1f0-77a5d406ddca admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:44.572683 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-8cc8141c-e09c-424d-a1f0-77a5d406ddca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0473a97c404c46138b9f8766e5a29c02', 'request_id': 'req-8cc8141c-e09c-424d-a1f0-77a5d406ddca'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:44.580398 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-8cc8141c-e09c-424d-a1f0-77a5d406ddca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0473a97c404c46138b9f8766e5a29c02', 'request_id': 'req-8cc8141c-e09c-424d-a1f0-77a5d406ddca'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:44.586839 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-8cc8141c-e09c-424d-a1f0-77a5d406ddca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0473a97c404c46138b9f8766e5a29c02', 'request_id': 'req-8cc8141c-e09c-424d-a1f0-77a5d406ddca'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:44.597864 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-8cc8141c-e09c-424d-a1f0-77a5d406ddca admin admin] The token cache is being invalidate because project 0473a97c404c46138b9f8766e5a29c02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:16:44.598152 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-8cc8141c-e09c-424d-a1f0-77a5d406ddca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:44.598824 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-8cc8141c-e09c-424d-a1f0-77a5d406ddca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:16:44.600373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1215/2416] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:16:44 2022] DELETE /identity/v3/projects/0473a97c404c46138b9f8766e5a29c02 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:16:44.605808 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887738a-3f83-4ad4-9be1-5167b3e46575 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:44.606096 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887738a-3f83-4ad4-9be1-5167b3e46575 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:44.606408 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887738a-3f83-4ad4-9be1-5167b3e46575 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:44.626237 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-a887738a-3f83-4ad4-9be1-5167b3e46575 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:16:44.627894 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a887738a-3f83-4ad4-9be1-5167b3e46575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:44.714598 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1202/2417] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:16:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:16:44.722994 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35475d89-c75b-41df-99b7-2751df4667c6 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:44.723652 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35475d89-c75b-41df-99b7-2751df4667c6 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:44.724187 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35475d89-c75b-41df-99b7-2751df4667c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:44.757593 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-35475d89-c75b-41df-99b7-2751df4667c6 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:16:44.758997 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-35475d89-c75b-41df-99b7-2751df4667c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:44.771530 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1216/2418] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:16:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:16:44.775652 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887738a-3f83-4ad4-9be1-5167b3e46575 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:44.786959 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a887738a-3f83-4ad4-9be1-5167b3e46575 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:44.793633 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e65f7-02e0-475e-958c-58063689824a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:44.794513 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e65f7-02e0-475e-958c-58063689824a admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:44.794851 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e65f7-02e0-475e-958c-58063689824a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:44.795235 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e65f7-02e0-475e-958c-58063689824a admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:44.796276 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e65f7-02e0-475e-958c-58063689824a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:16:44.796698 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e65f7-02e0-475e-958c-58063689824a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:44.804710 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e65f7-02e0-475e-958c-58063689824a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:44.827204 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1203/2419] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:16:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:16:44.831490 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35475d89-c75b-41df-99b7-2751df4667c6 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:44.849233 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35475d89-c75b-41df-99b7-2751df4667c6 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:44.855147 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3810af-0198-4e57-9737-3f9ca8b94953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:44.856036 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3810af-0198-4e57-9737-3f9ca8b94953 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:44.856479 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3810af-0198-4e57-9737-3f9ca8b94953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:44.856991 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3810af-0198-4e57-9737-3f9ca8b94953 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:44.857982 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3810af-0198-4e57-9737-3f9ca8b94953 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:44.860231 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3810af-0198-4e57-9737-3f9ca8b94953 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:44.898078 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1217/2420] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:16:44 2022] POST /identity/v3/projects => generated 375 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:16:44.902186 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e65f7-02e0-475e-958c-58063689824a admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:44.917416 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e65f7-02e0-475e-958c-58063689824a admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:45.014515 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ad7cf9-2516-4086-93fa-9816b786744e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:45.015296 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad7cf9-2516-4086-93fa-9816b786744e admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:45.015640 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad7cf9-2516-4086-93fa-9816b786744e admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:45.016039 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad7cf9-2516-4086-93fa-9816b786744e admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:45.016755 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ad7cf9-2516-4086-93fa-9816b786744e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:45.018909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ad7cf9-2516-4086-93fa-9816b786744e admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:45.043100 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1204/2421] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:16:44 2022] POST /identity/v3/users => generated 288 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:16:45.047403 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3810af-0198-4e57-9737-3f9ca8b94953 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:45.058203 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3810af-0198-4e57-9737-3f9ca8b94953 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:45.062963 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca2da5-688c-4bdd-995c-d4269a4e8432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:45.063578 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca2da5-688c-4bdd-995c-d4269a4e8432 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:45.063859 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca2da5-688c-4bdd-995c-d4269a4e8432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:45.064132 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca2da5-688c-4bdd-995c-d4269a4e8432 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:45.064780 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ca2da5-688c-4bdd-995c-d4269a4e8432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:45.066409 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ca2da5-688c-4bdd-995c-d4269a4e8432 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:45.077276 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1218/2422] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 20:16:45.081367 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ad7cf9-2516-4086-93fa-9816b786744e admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:45.092953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ad7cf9-2516-4086-93fa-9816b786744e admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:45.097672 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d899c2f-0c1f-495a-b7b3-d183b0f48bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:45.098303 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d899c2f-0c1f-495a-b7b3-d183b0f48bb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:45.098589 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d899c2f-0c1f-495a-b7b3-d183b0f48bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:45.098917 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d899c2f-0c1f-495a-b7b3-d183b0f48bb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:45.099502 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d899c2f-0c1f-495a-b7b3-d183b0f48bb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:45.101087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d899c2f-0c1f-495a-b7b3-d183b0f48bb9 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:45.112301 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1205/2423] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:16:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:16:45.116497 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca2da5-688c-4bdd-995c-d4269a4e8432 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:45.127447 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca2da5-688c-4bdd-995c-d4269a4e8432 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:45.131650 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c25824-e0da-426d-8dc9-7ff500bfe21a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:45.132337 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c25824-e0da-426d-8dc9-7ff500bfe21a admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:45.132617 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c25824-e0da-426d-8dc9-7ff500bfe21a admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:45.132900 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c25824-e0da-426d-8dc9-7ff500bfe21a admin admin] PATH_INFO: `/v3/projects/d6c53fa2e0e746de8a002871a359ae31/users/2f55a71c404c426b996f3dee3c2b92c5/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:45.148000 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c25824-e0da-426d-8dc9-7ff500bfe21a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6c53fa2e0e746de8a002871a359ae31, user_id=2f55a71c404c426b996f3dee3c2b92c5, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:45.151195 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c25824-e0da-426d-8dc9-7ff500bfe21a admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:45.165194 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1219/2424] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:16:45 2022] PUT /identity/v3/projects/d6c53fa2e0e746de8a002871a359ae31/users/2f55a71c404c426b996f3dee3c2b92c5/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:16:45.176415 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801aa20-8cda-4aa0-8b59-1d58cf7f85b7 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:45.176847 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801aa20-8cda-4aa0-8b59-1d58cf7f85b7 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:45.177129 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a801aa20-8cda-4aa0-8b59-1d58cf7f85b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:45.210024 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-a801aa20-8cda-4aa0-8b59-1d58cf7f85b7 None None] MFA Rules not processed for user `2f55a71c404c426b996f3dee3c2b92c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:16:45.211935 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a801aa20-8cda-4aa0-8b59-1d58cf7f85b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:45.223130 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:16:45.297333 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1206/2425] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:16:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:16:45.308238 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c25824-e0da-426d-8dc9-7ff500bfe21a admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:16:45.313178 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-27c25824-e0da-426d-8dc9-7ff500bfe21a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:45.384191 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c25824-e0da-426d-8dc9-7ff500bfe21a admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:16:45.388052 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a816b26-75d0-4635-9d64-5011ee927cd6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:16:45.388760 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a816b26-75d0-4635-9d64-5011ee927cd6 service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:45.389213 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a816b26-75d0-4635-9d64-5011ee927cd6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:45.389495 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a816b26-75d0-4635-9d64-5011ee927cd6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:45.395817 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:16:45.398192 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a816b26-75d0-4635-9d64-5011ee927cd6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:16:45.400285 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a816b26-75d0-4635-9d64-5011ee927cd6 service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:16:45.411584 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1220/2426] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:16:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:16:47.776247 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcbac7-1c06-4a5d-bc73-123f8679a0bc None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:16:47.776760 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcbac7-1c06-4a5d-bc73-123f8679a0bc None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:16:47.776880 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcbac7-1c06-4a5d-bc73-123f8679a0bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:16:47.795879 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-7ebcbac7-1c06-4a5d-bc73-123f8679a0bc None None] MFA Rules not processed for user `10bbbf111f714b108435b58c5b6c2747`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:16:47.797089 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7ebcbac7-1c06-4a5d-bc73-123f8679a0bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:16:47.849860 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1207/2427] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:16:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:16:55.412186 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=2 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:16:57.843424 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=47 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:17:00.779138 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a816b26-75d0-4635-9d64-5011ee927cd6 service cinder] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:17:00.847816 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a816b26-75d0-4635-9d64-5011ee927cd6 service cinder] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:17:00.852985 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3851cdf4-b690-4a11-86cb-14a31f6fd48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:17:00.853613 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851cdf4-b690-4a11-86cb-14a31f6fd48d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:17:00.853894 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851cdf4-b690-4a11-86cb-14a31f6fd48d admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:17:00.854168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3851cdf4-b690-4a11-86cb-14a31f6fd48d admin admin] PATH_INFO: `/v3/users/2f55a71c404c426b996f3dee3c2b92c5` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:17:00.855323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3851cdf4-b690-4a11-86cb-14a31f6fd48d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f55a71c404c426b996f3dee3c2b92c5)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:17:00.857256 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3851cdf4-b690-4a11-86cb-14a31f6fd48d admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:17:00.902492 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-3851cdf4-b690-4a11-86cb-14a31f6fd48d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f55a71c404c426b996f3dee3c2b92c5'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:17:00.914067 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-3851cdf4-b690-4a11-86cb-14a31f6fd48d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f55a71c404c426b996f3dee3c2b92c5'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:17:00.921489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-3851cdf4-b690-4a11-86cb-14a31f6fd48d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f55a71c404c426b996f3dee3c2b92c5'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:17:00.928517 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-3851cdf4-b690-4a11-86cb-14a31f6fd48d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f55a71c404c426b996f3dee3c2b92c5'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:17:00.929134 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-3851cdf4-b690-4a11-86cb-14a31f6fd48d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f55a71c404c426b996f3dee3c2b92c5'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:17:00.935102 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1221/2428] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:17:00 2022] DELETE /identity/v3/users/2f55a71c404c426b996f3dee3c2b92c5 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:17:00.939513 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310a7fd-eff5-462d-ad3c-26fb4b54a51b None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:17:00.940040 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310a7fd-eff5-462d-ad3c-26fb4b54a51b None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:17:00.940040 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3310a7fd-eff5-462d-ad3c-26fb4b54a51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:17:00.959749 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-3310a7fd-eff5-462d-ad3c-26fb4b54a51b None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:17:00.960933 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3310a7fd-eff5-462d-ad3c-26fb4b54a51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:17:01.018026 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1208/2429] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:17:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:17:01.028285 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3851cdf4-b690-4a11-86cb-14a31f6fd48d admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:17:01.029672 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-3851cdf4-b690-4a11-86cb-14a31f6fd48d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:17:01.087210 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3851cdf4-b690-4a11-86cb-14a31f6fd48d admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:17:01.090780 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504990ae-f0d9-4dcd-b449-1239d8c95745 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:17:01.091377 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504990ae-f0d9-4dcd-b449-1239d8c95745 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:17:01.091652 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504990ae-f0d9-4dcd-b449-1239d8c95745 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:17:01.091921 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504990ae-f0d9-4dcd-b449-1239d8c95745 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:17:01.100599 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504990ae-f0d9-4dcd-b449-1239d8c95745 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:17:01.102045 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504990ae-f0d9-4dcd-b449-1239d8c95745 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:17:01.111338 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1222/2430] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:17:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:17:01.454058 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310a7fd-eff5-462d-ad3c-26fb4b54a51b None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:17:01.455493 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3310a7fd-eff5-462d-ad3c-26fb4b54a51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:17:01.467950 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3310a7fd-eff5-462d-ad3c-26fb4b54a51b None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:17:01.472590 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d41081-85e4-4ab1-adc7-939c22ae76b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:17:01.473227 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d41081-85e4-4ab1-adc7-939c22ae76b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:17:01.473505 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d41081-85e4-4ab1-adc7-939c22ae76b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:17:01.473779 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d41081-85e4-4ab1-adc7-939c22ae76b8 admin admin] PATH_INFO: `/v3/projects/d6c53fa2e0e746de8a002871a359ae31` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:17:01.474998 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d41081-85e4-4ab1-adc7-939c22ae76b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6c53fa2e0e746de8a002871a359ae31)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:17:01.476978 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d41081-85e4-4ab1-adc7-939c22ae76b8 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:17:01.526013 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-84d41081-85e4-4ab1-adc7-939c22ae76b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6c53fa2e0e746de8a002871a359ae31', 'request_id': 'req-84d41081-85e4-4ab1-adc7-939c22ae76b8'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:17:01.532494 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-84d41081-85e4-4ab1-adc7-939c22ae76b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6c53fa2e0e746de8a002871a359ae31', 'request_id': 'req-84d41081-85e4-4ab1-adc7-939c22ae76b8'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:17:01.538158 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-84d41081-85e4-4ab1-adc7-939c22ae76b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6c53fa2e0e746de8a002871a359ae31', 'request_id': 'req-84d41081-85e4-4ab1-adc7-939c22ae76b8'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:17:01.549250 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-84d41081-85e4-4ab1-adc7-939c22ae76b8 admin admin] The token cache is being invalidate because project d6c53fa2e0e746de8a002871a359ae31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:17:01.549535 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-84d41081-85e4-4ab1-adc7-939c22ae76b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:17:01.550161 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-84d41081-85e4-4ab1-adc7-939c22ae76b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:17:01.551669 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1209/2431] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:17:01 2022] DELETE /identity/v3/projects/d6c53fa2e0e746de8a002871a359ae31 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:17:01.557701 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645cccb9-fc62-403e-9236-f54b618b094a None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:17:01.557987 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645cccb9-fc62-403e-9236-f54b618b094a None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:17:01.558287 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645cccb9-fc62-403e-9236-f54b618b094a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:17:01.578180 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-645cccb9-fc62-403e-9236-f54b618b094a None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:17:01.579385 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-645cccb9-fc62-403e-9236-f54b618b094a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:17:01.595501 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:17:01.638949 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1223/2432] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:17:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:17:01.646879 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0671e-97db-487d-8512-6143fb72cbc8 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:17:01.647199 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0671e-97db-487d-8512-6143fb72cbc8 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:17:01.647478 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e0671e-97db-487d-8512-6143fb72cbc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:17:01.667244 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-d8e0671e-97db-487d-8512-6143fb72cbc8 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:17:01.668464 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d8e0671e-97db-487d-8512-6143fb72cbc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:17:01.676462 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1210/2433] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:17:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:17:01.681023 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645cccb9-fc62-403e-9236-f54b618b094a None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:17:01.690675 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645cccb9-fc62-403e-9236-f54b618b094a None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:17:01.695222 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530dfcf6-2dee-4ed9-8f2c-6a2ee7298aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:17:01.696465 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530dfcf6-2dee-4ed9-8f2c-6a2ee7298aff admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:17:01.696714 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530dfcf6-2dee-4ed9-8f2c-6a2ee7298aff admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:17:01.696987 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530dfcf6-2dee-4ed9-8f2c-6a2ee7298aff admin admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:17:01.697678 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530dfcf6-2dee-4ed9-8f2c-6a2ee7298aff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:17:01.697981 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530dfcf6-2dee-4ed9-8f2c-6a2ee7298aff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:17:01.699786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530dfcf6-2dee-4ed9-8f2c-6a2ee7298aff admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:17:01.711427 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1224/2434] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:17:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:17:01.715519 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e0671e-97db-487d-8512-6143fb72cbc8 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:17:01.724765 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e0671e-97db-487d-8512-6143fb72cbc8 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:17:01.729406 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ddda0-6053-4895-ad00-babd1e62e766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:17:01.729997 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ddda0-6053-4895-ad00-babd1e62e766 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:17:01.730303 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ddda0-6053-4895-ad00-babd1e62e766 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:17:01.730579 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ddda0-6053-4895-ad00-babd1e62e766 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:17:01.731201 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ddda0-6053-4895-ad00-babd1e62e766 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:17:01.733196 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ddda0-6053-4895-ad00-babd1e62e766 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:17:01.760794 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1211/2435] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:17:01 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:17:01.765009 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530dfcf6-2dee-4ed9-8f2c-6a2ee7298aff admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:17:01.774841 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530dfcf6-2dee-4ed9-8f2c-6a2ee7298aff admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:17:01.829658 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed657ce9-220b-4d00-a929-30139333d7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:17:01.830478 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed657ce9-220b-4d00-a929-30139333d7da admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:17:01.830585 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed657ce9-220b-4d00-a929-30139333d7da admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:17:01.830818 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed657ce9-220b-4d00-a929-30139333d7da admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:17:01.831449 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed657ce9-220b-4d00-a929-30139333d7da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:17:01.833347 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed657ce9-220b-4d00-a929-30139333d7da admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:17:01.858033 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1225/2436] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:17:01 2022] POST /identity/v3/users => generated 290 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:17:01.859280 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ddda0-6053-4895-ad00-babd1e62e766 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:17:01.870324 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ddda0-6053-4895-ad00-babd1e62e766 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:17:01.874919 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365fab3-5eef-4b85-b287-dc51a27c2acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:17:01.875602 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365fab3-5eef-4b85-b287-dc51a27c2acc admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:17:01.875949 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365fab3-5eef-4b85-b287-dc51a27c2acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:17:01.876226 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365fab3-5eef-4b85-b287-dc51a27c2acc admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:17:01.877177 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365fab3-5eef-4b85-b287-dc51a27c2acc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:17:01.878969 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f365fab3-5eef-4b85-b287-dc51a27c2acc admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:17:01.889414 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1212/2437] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:17:01.893348 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed657ce9-220b-4d00-a929-30139333d7da admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:17:01.903168 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed657ce9-220b-4d00-a929-30139333d7da admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:17:01.907782 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d248c8-05f7-4320-af91-d4278c7f690d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:17:01.908373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d248c8-05f7-4320-af91-d4278c7f690d admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:17:01.908653 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d248c8-05f7-4320-af91-d4278c7f690d admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:17:01.908922 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d248c8-05f7-4320-af91-d4278c7f690d admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:17:01.909561 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d248c8-05f7-4320-af91-d4278c7f690d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:17:01.911170 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d248c8-05f7-4320-af91-d4278c7f690d admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:17:01.922118 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1226/2438] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:17:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:17:01.926158 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365fab3-5eef-4b85-b287-dc51a27c2acc admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:17:01.936451 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365fab3-5eef-4b85-b287-dc51a27c2acc admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:17:01.941022 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34253a89-353c-454b-a78e-fd8393cb25da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:17:01.941653 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34253a89-353c-454b-a78e-fd8393cb25da admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:17:01.942187 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34253a89-353c-454b-a78e-fd8393cb25da admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:17:01.942531 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34253a89-353c-454b-a78e-fd8393cb25da admin admin] PATH_INFO: `/v3/projects/deacebf04e2d4a8093425ce7d16ca3cf/users/2e15e1e499874c01b8d9b4aec9f9eb9c/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:17:01.957214 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34253a89-353c-454b-a78e-fd8393cb25da admin admin] RBAC: Authorizing `identity:create_grant(project_id=deacebf04e2d4a8093425ce7d16ca3cf, user_id=2e15e1e499874c01b8d9b4aec9f9eb9c, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:17:01.959840 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34253a89-353c-454b-a78e-fd8393cb25da admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:17:01.972239 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1213/2439] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:17:01 2022] PUT /identity/v3/projects/deacebf04e2d4a8093425ce7d16ca3cf/users/2e15e1e499874c01b8d9b4aec9f9eb9c/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:17:01.977707 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6982a9f-dd5c-4c98-ac00-570d1e42f06e None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:17:01.977989 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6982a9f-dd5c-4c98-ac00-570d1e42f06e None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:17:01.978270 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6982a9f-dd5c-4c98-ac00-570d1e42f06e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:17:01.997748 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-d6982a9f-dd5c-4c98-ac00-570d1e42f06e None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:17:01.999034 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-d6982a9f-dd5c-4c98-ac00-570d1e42f06e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:17:02.057478 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1227/2440] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:17:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:17:02.067028 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34253a89-353c-454b-a78e-fd8393cb25da admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:17:02.068750 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-34253a89-353c-454b-a78e-fd8393cb25da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:17:02.090565 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:17:02.131997 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34253a89-353c-454b-a78e-fd8393cb25da admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:17:02.135676 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c842a-8434-47f2-8d5e-316cd9c286e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:17:02.136282 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c842a-8434-47f2-8d5e-316cd9c286e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:17:02.136561 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c842a-8434-47f2-8d5e-316cd9c286e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:17:02.136834 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485c842a-8434-47f2-8d5e-316cd9c286e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:17:02.146084 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485c842a-8434-47f2-8d5e-316cd9c286e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:17:02.147595 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485c842a-8434-47f2-8d5e-316cd9c286e3 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:17:02.157064 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1214/2441] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:17:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 20:17:06.112226 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e715a-2e00-4b28-978c-2e3edcd29769 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:17:06.112718 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e715a-2e00-4b28-978c-2e3edcd29769 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:17:06.112871 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4e715a-2e00-4b28-978c-2e3edcd29769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:17:06.153899 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-8b4e715a-2e00-4b28-978c-2e3edcd29769 None None] MFA Rules not processed for user `2e15e1e499874c01b8d9b4aec9f9eb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:17:06.155256 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-8b4e715a-2e00-4b28-978c-2e3edcd29769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:17:06.175584 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:17:06.240075 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1228/2442] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:17:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:17:06.245179 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c842a-8434-47f2-8d5e-316cd9c286e3 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:17:06.255336 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485c842a-8434-47f2-8d5e-316cd9c286e3 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:17:06.260862 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e6594-53d6-48e7-b5cc-8d16f356d731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:17:06.261527 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e6594-53d6-48e7-b5cc-8d16f356d731 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:17:06.261784 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e6594-53d6-48e7-b5cc-8d16f356d731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:17:06.262047 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52e6594-53d6-48e7-b5cc-8d16f356d731 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:17:06.262721 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52e6594-53d6-48e7-b5cc-8d16f356d731 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:17:06.264654 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52e6594-53d6-48e7-b5cc-8d16f356d731 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:17:06.292396 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1215/2443] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:17:06 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:17:06.296746 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e715a-2e00-4b28-978c-2e3edcd29769 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:17:06.308097 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4e715a-2e00-4b28-978c-2e3edcd29769 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:17:06.366867 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000a0c3-28d4-465d-98bf-9c74ab76b5bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:17:06.369190 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000a0c3-28d4-465d-98bf-9c74ab76b5bc admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:17:06.369481 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000a0c3-28d4-465d-98bf-9c74ab76b5bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:17:06.369761 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4000a0c3-28d4-465d-98bf-9c74ab76b5bc admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:17:06.370441 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4000a0c3-28d4-465d-98bf-9c74ab76b5bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:17:06.372371 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4000a0c3-28d4-465d-98bf-9c74ab76b5bc admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:17:06.393787 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1229/2444] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:17:06 2022] POST /identity/v3/users => generated 295 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:17:06.397499 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e6594-53d6-48e7-b5cc-8d16f356d731 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:17:06.407630 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52e6594-53d6-48e7-b5cc-8d16f356d731 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:17:06.412103 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d4ec2-4e7c-4397-9bee-4d8dac26b8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:17:06.412708 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d4ec2-4e7c-4397-9bee-4d8dac26b8e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:17:06.412982 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d4ec2-4e7c-4397-9bee-4d8dac26b8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:17:06.413258 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d4ec2-4e7c-4397-9bee-4d8dac26b8e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:17:06.413903 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65d4ec2-4e7c-4397-9bee-4d8dac26b8e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:17:06.416098 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65d4ec2-4e7c-4397-9bee-4d8dac26b8e6 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:17:06.426696 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1216/2445] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:17:06.431345 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000a0c3-28d4-465d-98bf-9c74ab76b5bc admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:17:06.441761 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4000a0c3-28d4-465d-98bf-9c74ab76b5bc admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:17:06.446521 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19aa9d-7aaf-46ae-9ed6-49b3ece330fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:17:06.447247 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19aa9d-7aaf-46ae-9ed6-49b3ece330fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:17:06.447548 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19aa9d-7aaf-46ae-9ed6-49b3ece330fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:17:06.447873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19aa9d-7aaf-46ae-9ed6-49b3ece330fe admin admin] PATH_INFO: `/v3/projects/1b7ac947dae84984adc4c779cb78e4c3/users/958f84e23a744656b292d68702721ed5/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:17:06.463204 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19aa9d-7aaf-46ae-9ed6-49b3ece330fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b7ac947dae84984adc4c779cb78e4c3, user_id=958f84e23a744656b292d68702721ed5, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:17:06.465873 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19aa9d-7aaf-46ae-9ed6-49b3ece330fe admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:17:06.480201 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1230/2446] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:17:06 2022] PUT /identity/v3/projects/1b7ac947dae84984adc4c779cb78e4c3/users/958f84e23a744656b292d68702721ed5/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:17:06.484199 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d4ec2-4e7c-4397-9bee-4d8dac26b8e6 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:17:06.494983 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d4ec2-4e7c-4397-9bee-4d8dac26b8e6 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:17:06.554677 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e281b-3388-4542-ba2a-18f830587a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:17:06.555592 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e281b-3388-4542-ba2a-18f830587a17 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:17:06.555859 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e281b-3388-4542-ba2a-18f830587a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:17:06.556131 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26e281b-3388-4542-ba2a-18f830587a17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:17:06.557641 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e281b-3388-4542-ba2a-18f830587a17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:17:06.559305 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26e281b-3388-4542-ba2a-18f830587a17 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:17:06.573219 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1217/2447] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:17:06 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:17:06.574969 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19aa9d-7aaf-46ae-9ed6-49b3ece330fe admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:17:06.587858 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19aa9d-7aaf-46ae-9ed6-49b3ece330fe admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:17:06.592711 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e22aa0e-c0a2-4984-9ddb-4e75d7fab4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:17:06.593388 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22aa0e-c0a2-4984-9ddb-4e75d7fab4c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:17:06.593669 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22aa0e-c0a2-4984-9ddb-4e75d7fab4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:17:06.593945 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e22aa0e-c0a2-4984-9ddb-4e75d7fab4c4 admin admin] PATH_INFO: `/v3/projects/1b7ac947dae84984adc4c779cb78e4c3/users/958f84e23a744656b292d68702721ed5/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:17:06.596005 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e22aa0e-c0a2-4984-9ddb-4e75d7fab4c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b7ac947dae84984adc4c779cb78e4c3, user_id=958f84e23a744656b292d68702721ed5, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:17:06.598723 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e22aa0e-c0a2-4984-9ddb-4e75d7fab4c4 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:17:06.610413 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1231/2448] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:17:06 2022] PUT /identity/v3/projects/1b7ac947dae84984adc4c779cb78e4c3/users/958f84e23a744656b292d68702721ed5/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:17:06.953186 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e281b-3388-4542-ba2a-18f830587a17 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:17:06.955030 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a26e281b-3388-4542-ba2a-18f830587a17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:17:07.085081 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26e281b-3388-4542-ba2a-18f830587a17 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:17:07.110117 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:17:07.184064 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b306b60-c25f-42cf-aa48-a23dd7f5d2bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:17:07.184772 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b306b60-c25f-42cf-aa48-a23dd7f5d2bc service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:17:07.185135 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b306b60-c25f-42cf-aa48-a23dd7f5d2bc service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:17:07.185473 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b306b60-c25f-42cf-aa48-a23dd7f5d2bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:17:07.187590 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9b306b60-c25f-42cf-aa48-a23dd7f5d2bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:17:07.201432 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b306b60-c25f-42cf-aa48-a23dd7f5d2bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:17:07.203198 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b306b60-c25f-42cf-aa48-a23dd7f5d2bc service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:17:07.219335 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1218/2449] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:17:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:17:11.772829 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1dde3-c3c1-45bb-86a0-9324c0068c33 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:17:11.773288 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1dde3-c3c1-45bb-86a0-9324c0068c33 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:17:11.773434 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd1dde3-c3c1-45bb-86a0-9324c0068c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:17:11.808332 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-9fd1dde3-c3c1-45bb-86a0-9324c0068c33 None None] MFA Rules not processed for user `958f84e23a744656b292d68702721ed5`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:17:11.809580 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-9fd1dde3-c3c1-45bb-86a0-9324c0068c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:17:11.887922 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:17:11.901542 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1232/2450] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:17:11 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:17:11.915671 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b306b60-c25f-42cf-aa48-a23dd7f5d2bc service nova] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:17:11.917684 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9b306b60-c25f-42cf-aa48-a23dd7f5d2bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:17:11.934264 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b306b60-c25f-42cf-aa48-a23dd7f5d2bc service nova] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:17:11.941832 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc839a8c-5b3b-4a82-921c-90959937aa33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:17:11.942793 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc839a8c-5b3b-4a82-921c-90959937aa33 service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:17:11.943183 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc839a8c-5b3b-4a82-921c-90959937aa33 service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:17:11.943727 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc839a8c-5b3b-4a82-921c-90959937aa33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:17:12.018163 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc839a8c-5b3b-4a82-921c-90959937aa33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:17:12.021520 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc839a8c-5b3b-4a82-921c-90959937aa33 service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:17:12.036371 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1219/2451] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:17:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:17:13.717557 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd1dde3-c3c1-45bb-86a0-9324c0068c33 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:17:13.733438 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd1dde3-c3c1-45bb-86a0-9324c0068c33 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:17:13.745852 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d831399-b439-45a9-9e51-49c3906f7575 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:17:13.746656 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d831399-b439-45a9-9e51-49c3906f7575 service nova] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:17:13.746946 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d831399-b439-45a9-9e51-49c3906f7575 service nova] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:17:13.747218 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d831399-b439-45a9-9e51-49c3906f7575 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:17:13.749050 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-4d831399-b439-45a9-9e51-49c3906f7575 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:17:13.789588 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d831399-b439-45a9-9e51-49c3906f7575 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:17:13.791481 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d831399-b439-45a9-9e51-49c3906f7575 service nova] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:17:13.806779 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1233/2452] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:17:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:17:22.030803 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=51 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:17:23.800963 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=8 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:17:29.982161 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe62529-9b9e-4eb9-aa1b-8de97620c142 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:17:29.982677 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe62529-9b9e-4eb9-aa1b-8de97620c142 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:17:29.982771 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe62529-9b9e-4eb9-aa1b-8de97620c142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:17:30.005585 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-cfe62529-9b9e-4eb9-aa1b-8de97620c142 None None] MFA Rules not processed for user `52c576b9f49b4900ac5ba707aba78c27`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:17:30.007005 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-cfe62529-9b9e-4eb9-aa1b-8de97620c142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:17:30.065189 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1220/2453] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:17:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:17:30.295223 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d831399-b439-45a9-9e51-49c3906f7575 service nova] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:17:30.348663 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d831399-b439-45a9-9e51-49c3906f7575 service nova] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:17:30.352296 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a64564-c61a-4c97-926f-4741ebb3cd22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:17:30.352895 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a64564-c61a-4c97-926f-4741ebb3cd22 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:17:30.353172 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a64564-c61a-4c97-926f-4741ebb3cd22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:17:30.353445 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a64564-c61a-4c97-926f-4741ebb3cd22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:17:30.355155 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-32a64564-c61a-4c97-926f-4741ebb3cd22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:17:30.364936 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a64564-c61a-4c97-926f-4741ebb3cd22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:17:30.366385 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a64564-c61a-4c97-926f-4741ebb3cd22 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:17:30.374870 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1234/2454] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:17:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:17:30.681726 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe62529-9b9e-4eb9-aa1b-8de97620c142 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:17:30.683281 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-cfe62529-9b9e-4eb9-aa1b-8de97620c142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:17:30.752690 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe62529-9b9e-4eb9-aa1b-8de97620c142 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:17:30.757627 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b1bcad-f206-4d42-83d9-7980d4c11919 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312489c49a304ca9aacebd9843a4b65f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:17:30.758225 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1bcad-f206-4d42-83d9-7980d4c11919 service placement] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:17:30.758537 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1bcad-f206-4d42-83d9-7980d4c11919 service placement] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:17:30.758809 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b1bcad-f206-4d42-83d9-7980d4c11919 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:17:30.760516 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-84b1bcad-f206-4d42-83d9-7980d4c11919 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:17:30.771706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b1bcad-f206-4d42-83d9-7980d4c11919 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:17:30.773264 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b1bcad-f206-4d42-83d9-7980d4c11919 service placement] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:17:30.783629 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1221/2455] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:17:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:17:39.598066 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb523e3-a782-4d90-bc4a-7581a3cac7bb None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:17:39.598705 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb523e3-a782-4d90-bc4a-7581a3cac7bb None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:17:39.598705 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb523e3-a782-4d90-bc4a-7581a3cac7bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:17:39.617648 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-8cb523e3-a782-4d90-bc4a-7581a3cac7bb None None] MFA Rules not processed for user `2e15e1e499874c01b8d9b4aec9f9eb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:17:39.618851 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-8cb523e3-a782-4d90-bc4a-7581a3cac7bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:17:39.625540 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1235/2456] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:17:39 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:17:40.778810 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=57 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:17:49.612295 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=29 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:18:05.449255 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b1bcad-f206-4d42-83d9-7980d4c11919 service placement] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:18:05.458860 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b1bcad-f206-4d42-83d9-7980d4c11919 service placement] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:18:05.462422 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93122773-0cd8-4006-b4ac-7640470515ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:18:05.463013 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93122773-0cd8-4006-b4ac-7640470515ff service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:18:05.463325 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93122773-0cd8-4006-b4ac-7640470515ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:18:05.463599 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93122773-0cd8-4006-b4ac-7640470515ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:18:05.472279 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93122773-0cd8-4006-b4ac-7640470515ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:18:05.473681 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93122773-0cd8-4006-b4ac-7640470515ff service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:18:05.483655 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1222/2457] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:18:05 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:18:15.478751 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=6 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:18:29.627645 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb523e3-a782-4d90-bc4a-7581a3cac7bb None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:18:29.639451 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb523e3-a782-4d90-bc4a-7581a3cac7bb None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:18:29.644458 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67db836-75a3-4336-96b7-a7baf58d3473 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312489c49a304ca9aacebd9843a4b65f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:18:29.645086 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67db836-75a3-4336-96b7-a7baf58d3473 service placement] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:18:29.645369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67db836-75a3-4336-96b7-a7baf58d3473 service placement] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:18:29.645646 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67db836-75a3-4336-96b7-a7baf58d3473 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:18:29.647405 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-a67db836-75a3-4336-96b7-a7baf58d3473 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:18:29.658686 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67db836-75a3-4336-96b7-a7baf58d3473 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:18:29.660253 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67db836-75a3-4336-96b7-a7baf58d3473 service placement] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:18:29.670027 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1236/2458] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:18:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:18:37.393737 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93122773-0cd8-4006-b4ac-7640470515ff service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:18:37.395406 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-93122773-0cd8-4006-b4ac-7640470515ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:18:37.460180 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93122773-0cd8-4006-b4ac-7640470515ff service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:18:37.464535 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d43c6-a954-4e7c-a519-f7b39695f16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:18:37.465148 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d43c6-a954-4e7c-a519-f7b39695f16a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:18:37.465424 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d43c6-a954-4e7c-a519-f7b39695f16a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:18:37.465693 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495d43c6-a954-4e7c-a519-f7b39695f16a admin admin] PATH_INFO: `/v3/users/52c576b9f49b4900ac5ba707aba78c27` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:18:37.466812 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d43c6-a954-4e7c-a519-f7b39695f16a admin admin] RBAC: Authorizing `identity:delete_user(user_id=52c576b9f49b4900ac5ba707aba78c27)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:18:37.468795 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495d43c6-a954-4e7c-a519-f7b39695f16a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:18:37.513295 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-495d43c6-a954-4e7c-a519-f7b39695f16a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52c576b9f49b4900ac5ba707aba78c27'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:18:37.525581 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-495d43c6-a954-4e7c-a519-f7b39695f16a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52c576b9f49b4900ac5ba707aba78c27'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:18:37.532144 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-495d43c6-a954-4e7c-a519-f7b39695f16a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52c576b9f49b4900ac5ba707aba78c27'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:18:37.532835 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-495d43c6-a954-4e7c-a519-f7b39695f16a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52c576b9f49b4900ac5ba707aba78c27'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:18:37.539797 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-495d43c6-a954-4e7c-a519-f7b39695f16a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52c576b9f49b4900ac5ba707aba78c27'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:18:37.545955 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1223/2459] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:18:37 2022] DELETE /identity/v3/users/52c576b9f49b4900ac5ba707aba78c27 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:18:37.719169 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67db836-75a3-4336-96b7-a7baf58d3473 service placement] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:18:37.730393 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67db836-75a3-4336-96b7-a7baf58d3473 service placement] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:18:37.795415 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce210ecc-82a2-4dc0-9b58-f9228f4ea739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:18:37.797000 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce210ecc-82a2-4dc0-9b58-f9228f4ea739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:18:37.797000 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce210ecc-82a2-4dc0-9b58-f9228f4ea739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:18:37.797219 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce210ecc-82a2-4dc0-9b58-f9228f4ea739 admin admin] PATH_INFO: `/v3/projects/0df26e575de34eda9d416de65ec13036` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:18:37.798221 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce210ecc-82a2-4dc0-9b58-f9228f4ea739 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0df26e575de34eda9d416de65ec13036)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:18:37.800128 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce210ecc-82a2-4dc0-9b58-f9228f4ea739 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:18:37.848434 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ce210ecc-82a2-4dc0-9b58-f9228f4ea739 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0df26e575de34eda9d416de65ec13036', 'request_id': 'req-ce210ecc-82a2-4dc0-9b58-f9228f4ea739'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:18:37.856849 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ce210ecc-82a2-4dc0-9b58-f9228f4ea739 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0df26e575de34eda9d416de65ec13036', 'request_id': 'req-ce210ecc-82a2-4dc0-9b58-f9228f4ea739'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:18:37.862755 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ce210ecc-82a2-4dc0-9b58-f9228f4ea739 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0df26e575de34eda9d416de65ec13036', 'request_id': 'req-ce210ecc-82a2-4dc0-9b58-f9228f4ea739'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:18:37.876406 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ce210ecc-82a2-4dc0-9b58-f9228f4ea739 admin admin] The token cache is being invalidate because project 0df26e575de34eda9d416de65ec13036 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:18:37.876698 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ce210ecc-82a2-4dc0-9b58-f9228f4ea739 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:18:37.877597 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ce210ecc-82a2-4dc0-9b58-f9228f4ea739 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:18:37.879132 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1237/2460] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:18:37 2022] DELETE /identity/v3/projects/0df26e575de34eda9d416de65ec13036 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:18:47.542407 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:18:47.871673 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:18:54.206699 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d43c6-a954-4e7c-a519-f7b39695f16a admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:18:54.208402 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-495d43c6-a954-4e7c-a519-f7b39695f16a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:18:54.281247 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495d43c6-a954-4e7c-a519-f7b39695f16a admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:18:54.341367 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc0d143-f4f7-422f-909f-105af0984cbf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:18:54.341963 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0d143-f4f7-422f-909f-105af0984cbf service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:18:54.342238 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0d143-f4f7-422f-909f-105af0984cbf service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:18:54.343082 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc0d143-f4f7-422f-909f-105af0984cbf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:18:54.344782 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5bc0d143-f4f7-422f-909f-105af0984cbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:18:54.403730 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc0d143-f4f7-422f-909f-105af0984cbf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:18:54.405945 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc0d143-f4f7-422f-909f-105af0984cbf service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:18:54.414875 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1224/2461] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:18:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:19:04.410705 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=84 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:19:12.003419 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14dad20-cf9e-4533-9344-a8b1b12b94c9 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:19:12.003919 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14dad20-cf9e-4533-9344-a8b1b12b94c9 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:19:12.004031 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14dad20-cf9e-4533-9344-a8b1b12b94c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:19:12.023478 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-f14dad20-cf9e-4533-9344-a8b1b12b94c9 None None] MFA Rules not processed for user `10bbbf111f714b108435b58c5b6c2747`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:19:12.024752 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-f14dad20-cf9e-4533-9344-a8b1b12b94c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:19:12.032139 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1238/2462] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:19:12 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:19:12.329809 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc0d143-f4f7-422f-909f-105af0984cbf service nova] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:19:12.331205 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5bc0d143-f4f7-422f-909f-105af0984cbf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:19:12.386439 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc0d143-f4f7-422f-909f-105af0984cbf service nova] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:19:12.389922 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b1dc9e-7533-4926-8508-dcef41084eb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:19:12.390543 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1dc9e-7533-4926-8508-dcef41084eb0 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:19:12.390819 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1dc9e-7533-4926-8508-dcef41084eb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:19:12.391087 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b1dc9e-7533-4926-8508-dcef41084eb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:19:12.392788 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-b5b1dc9e-7533-4926-8508-dcef41084eb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:19:12.402096 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b1dc9e-7533-4926-8508-dcef41084eb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:19:12.403546 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b1dc9e-7533-4926-8508-dcef41084eb0 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:19:12.411431 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1225/2463] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:19:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:19:22.018339 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:19:22.407833 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=26 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:20:28.984374 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5c2a7-d7d8-472a-b239-6610de7773d5 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:20:28.984875 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5c2a7-d7d8-472a-b239-6610de7773d5 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:20:28.984974 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5c2a7-d7d8-472a-b239-6610de7773d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:20:29.013407 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-73d5c2a7-d7d8-472a-b239-6610de7773d5 None None] MFA Rules not processed for user `2e15e1e499874c01b8d9b4aec9f9eb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:20:29.014754 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-73d5c2a7-d7d8-472a-b239-6610de7773d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:20:29.073471 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1239/2464] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:20:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:20:30.822004 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b1dc9e-7533-4926-8508-dcef41084eb0 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:20:30.831843 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b1dc9e-7533-4926-8508-dcef41084eb0 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:20:30.835751 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dde45f-560c-4c85-a4b7-285595525a1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:20:30.836347 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dde45f-560c-4c85-a4b7-285595525a1a service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:20:30.836624 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dde45f-560c-4c85-a4b7-285595525a1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:20:30.836894 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dde45f-560c-4c85-a4b7-285595525a1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:20:30.838622 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-32dde45f-560c-4c85-a4b7-285595525a1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:20:30.852117 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dde45f-560c-4c85-a4b7-285595525a1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:20:30.853530 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dde45f-560c-4c85-a4b7-285595525a1a service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:20:30.864417 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1226/2465] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:20:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:20:32.495286 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5c2a7-d7d8-472a-b239-6610de7773d5 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:20:32.496697 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-73d5c2a7-d7d8-472a-b239-6610de7773d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:20:32.554033 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5c2a7-d7d8-472a-b239-6610de7773d5 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:20:32.557634 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5b0ff3-10a0-4ca7-8c33-b01194143e98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:20:32.558240 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b0ff3-10a0-4ca7-8c33-b01194143e98 service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:20:32.558539 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b0ff3-10a0-4ca7-8c33-b01194143e98 service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:20:32.558808 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b0ff3-10a0-4ca7-8c33-b01194143e98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:20:32.560513 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-5a5b0ff3-10a0-4ca7-8c33-b01194143e98 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:20:32.574092 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5b0ff3-10a0-4ca7-8c33-b01194143e98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:20:32.575872 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5b0ff3-10a0-4ca7-8c33-b01194143e98 service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:20:32.587826 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1240/2466] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:20:32 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:20:40.859525 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=6 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:20:42.582636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=51 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:20:49.972995 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dde45f-560c-4c85-a4b7-285595525a1a service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:20:49.981657 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dde45f-560c-4c85-a4b7-285595525a1a service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:20:49.985535 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca11da8c-a9b9-4ac3-be2d-1836fc219359 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:20:49.986143 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca11da8c-a9b9-4ac3-be2d-1836fc219359 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:20:49.986470 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca11da8c-a9b9-4ac3-be2d-1836fc219359 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:20:49.986747 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca11da8c-a9b9-4ac3-be2d-1836fc219359 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:20:49.988499 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ca11da8c-a9b9-4ac3-be2d-1836fc219359 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:20:50.053483 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca11da8c-a9b9-4ac3-be2d-1836fc219359 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:20:50.054954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca11da8c-a9b9-4ac3-be2d-1836fc219359 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:20:50.064579 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1227/2467] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:20:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:20:53.241897 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5b0ff3-10a0-4ca7-8c33-b01194143e98 service cinder] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:20:53.244057 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-5a5b0ff3-10a0-4ca7-8c33-b01194143e98 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:20:53.304252 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca11da8c-a9b9-4ac3-be2d-1836fc219359 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:20:53.305845 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-ca11da8c-a9b9-4ac3-be2d-1836fc219359 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:20:53.325753 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca11da8c-a9b9-4ac3-be2d-1836fc219359 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:20:53.331681 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c1a16-4117-40ef-ba72-d05fb24dad30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:20:53.332324 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c1a16-4117-40ef-ba72-d05fb24dad30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:20:53.332614 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c1a16-4117-40ef-ba72-d05fb24dad30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:20:53.332722 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5b0ff3-10a0-4ca7-8c33-b01194143e98 service cinder] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:20:53.332887 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c1a16-4117-40ef-ba72-d05fb24dad30 admin admin] PATH_INFO: `/v3/users/10bbbf111f714b108435b58c5b6c2747` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:20:53.334003 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c1a16-4117-40ef-ba72-d05fb24dad30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10bbbf111f714b108435b58c5b6c2747)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:20:53.335990 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c1a16-4117-40ef-ba72-d05fb24dad30 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:20:53.337778 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75b665-e1c6-48f5-aefe-2748f7b80a35 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312489c49a304ca9aacebd9843a4b65f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:20:53.338424 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75b665-e1c6-48f5-aefe-2748f7b80a35 service placement] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:20:53.338714 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75b665-e1c6-48f5-aefe-2748f7b80a35 service placement] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:20:53.339013 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75b665-e1c6-48f5-aefe-2748f7b80a35 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:20:53.340891 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-cc75b665-e1c6-48f5-aefe-2748f7b80a35 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:20:53.353061 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc75b665-e1c6-48f5-aefe-2748f7b80a35 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:20:53.354703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc75b665-e1c6-48f5-aefe-2748f7b80a35 service placement] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:20:53.365648 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1241/2468] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:20:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:20:53.389312 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5e6c1a16-4117-40ef-ba72-d05fb24dad30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10bbbf111f714b108435b58c5b6c2747'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:20:53.402898 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5e6c1a16-4117-40ef-ba72-d05fb24dad30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10bbbf111f714b108435b58c5b6c2747'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:20:53.410791 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5e6c1a16-4117-40ef-ba72-d05fb24dad30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10bbbf111f714b108435b58c5b6c2747'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:20:53.411476 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5e6c1a16-4117-40ef-ba72-d05fb24dad30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10bbbf111f714b108435b58c5b6c2747'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:20:53.418719 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5e6c1a16-4117-40ef-ba72-d05fb24dad30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10bbbf111f714b108435b58c5b6c2747'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:20:53.425271 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1228/2469] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:20:53 2022] DELETE /identity/v3/users/10bbbf111f714b108435b58c5b6c2747 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:20:53.598724 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75b665-e1c6-48f5-aefe-2748f7b80a35 service placement] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:20:53.611668 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75b665-e1c6-48f5-aefe-2748f7b80a35 service placement] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:20:53.668407 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72076b2b-e4fd-4919-b9e1-bc73bd13c09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:20:53.669042 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72076b2b-e4fd-4919-b9e1-bc73bd13c09b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:20:53.669324 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72076b2b-e4fd-4919-b9e1-bc73bd13c09b admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:20:53.669595 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72076b2b-e4fd-4919-b9e1-bc73bd13c09b admin admin] PATH_INFO: `/v3/projects/71f456c92d544d0c81bcb2fe937188ef` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:20:53.670761 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72076b2b-e4fd-4919-b9e1-bc73bd13c09b admin admin] RBAC: Authorizing `identity:delete_project(project_id=71f456c92d544d0c81bcb2fe937188ef)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:20:53.672742 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72076b2b-e4fd-4919-b9e1-bc73bd13c09b admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:20:53.718863 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-72076b2b-e4fd-4919-b9e1-bc73bd13c09b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71f456c92d544d0c81bcb2fe937188ef', 'request_id': 'req-72076b2b-e4fd-4919-b9e1-bc73bd13c09b'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:20:53.726693 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-72076b2b-e4fd-4919-b9e1-bc73bd13c09b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71f456c92d544d0c81bcb2fe937188ef', 'request_id': 'req-72076b2b-e4fd-4919-b9e1-bc73bd13c09b'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:20:53.732275 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-72076b2b-e4fd-4919-b9e1-bc73bd13c09b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71f456c92d544d0c81bcb2fe937188ef', 'request_id': 'req-72076b2b-e4fd-4919-b9e1-bc73bd13c09b'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:20:53.743450 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-72076b2b-e4fd-4919-b9e1-bc73bd13c09b admin admin] The token cache is being invalidate because project 71f456c92d544d0c81bcb2fe937188ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:20:53.743734 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-72076b2b-e4fd-4919-b9e1-bc73bd13c09b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:20:53.744370 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-72076b2b-e4fd-4919-b9e1-bc73bd13c09b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:20:53.745815 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1242/2470] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:20:53 2022] DELETE /identity/v3/projects/71f456c92d544d0c81bcb2fe937188ef => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:20:53.750763 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55ecab-6b5e-4c6e-b0c6-11699f0a45e6 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:20:53.751052 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55ecab-6b5e-4c6e-b0c6-11699f0a45e6 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:20:53.751326 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd55ecab-6b5e-4c6e-b0c6-11699f0a45e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:20:53.771135 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-bd55ecab-6b5e-4c6e-b0c6-11699f0a45e6 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:20:53.772270 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-bd55ecab-6b5e-4c6e-b0c6-11699f0a45e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:20:53.833621 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1229/2471] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:20:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:20:53.841321 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bc32e-d78e-439c-b372-0ba5097c4ce7 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:20:53.841607 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bc32e-d78e-439c-b372-0ba5097c4ce7 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:20:53.841881 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3bc32e-d78e-439c-b372-0ba5097c4ce7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:20:53.863294 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-bf3bc32e-d78e-439c-b372-0ba5097c4ce7 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:20:53.864520 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-bf3bc32e-d78e-439c-b372-0ba5097c4ce7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:20:53.872665 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1243/2472] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:20:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:20:53.877001 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55ecab-6b5e-4c6e-b0c6-11699f0a45e6 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:20:53.886793 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd55ecab-6b5e-4c6e-b0c6-11699f0a45e6 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:20:53.891908 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d33d2d5-6217-44fd-880a-78596b39c5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:20:53.892524 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33d2d5-6217-44fd-880a-78596b39c5dd admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:20:53.892837 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33d2d5-6217-44fd-880a-78596b39c5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:20:53.893104 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d33d2d5-6217-44fd-880a-78596b39c5dd admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:20:53.893851 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d33d2d5-6217-44fd-880a-78596b39c5dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:20:53.894238 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d33d2d5-6217-44fd-880a-78596b39c5dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:20:53.895966 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d33d2d5-6217-44fd-880a-78596b39c5dd admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:20:53.897528 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:20:53.910128 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1230/2473] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:20:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:20:53.914341 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3bc32e-d78e-439c-b372-0ba5097c4ce7 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:20:53.923561 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3bc32e-d78e-439c-b372-0ba5097c4ce7 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:20:53.928248 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bcc99b-0854-4d6c-85f3-f68464935e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:20:53.928843 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bcc99b-0854-4d6c-85f3-f68464935e72 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:20:53.929122 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bcc99b-0854-4d6c-85f3-f68464935e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:20:53.929391 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bcc99b-0854-4d6c-85f3-f68464935e72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:20:53.930015 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bcc99b-0854-4d6c-85f3-f68464935e72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:20:53.931937 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bcc99b-0854-4d6c-85f3-f68464935e72 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:20:53.943902 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:20:53.961899 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1244/2474] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:20:53 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:20:53.965953 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d33d2d5-6217-44fd-880a-78596b39c5dd admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:20:53.976789 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d33d2d5-6217-44fd-880a-78596b39c5dd admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:20:54.032438 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b3228-4462-48ea-9605-32d30af4bd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:20:54.033041 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b3228-4462-48ea-9605-32d30af4bd5c admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:20:54.033317 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b3228-4462-48ea-9605-32d30af4bd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:20:54.033586 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b3228-4462-48ea-9605-32d30af4bd5c admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:20:54.034221 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1b3228-4462-48ea-9605-32d30af4bd5c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:20:54.036136 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1b3228-4462-48ea-9605-32d30af4bd5c admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:20:54.056443 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1231/2475] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:20:53 2022] POST /identity/v3/users => generated 283 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:20:54.060975 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bcc99b-0854-4d6c-85f3-f68464935e72 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:20:54.072047 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bcc99b-0854-4d6c-85f3-f68464935e72 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:20:54.076962 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe4036-8e37-4abb-818c-46f93109c3d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:20:54.077559 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe4036-8e37-4abb-818c-46f93109c3d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:20:54.077839 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe4036-8e37-4abb-818c-46f93109c3d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:20:54.078111 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe4036-8e37-4abb-818c-46f93109c3d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:20:54.079376 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe4036-8e37-4abb-818c-46f93109c3d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:20:54.081001 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe4036-8e37-4abb-818c-46f93109c3d9 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:20:54.090958 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1245/2476] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:20:54.094900 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b3228-4462-48ea-9605-32d30af4bd5c admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:20:54.104621 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b3228-4462-48ea-9605-32d30af4bd5c admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:20:54.109197 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbbd6a4-ce87-42bf-8b2d-85f8fd58140a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:20:54.109809 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbd6a4-ce87-42bf-8b2d-85f8fd58140a admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:20:54.110091 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbd6a4-ce87-42bf-8b2d-85f8fd58140a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:20:54.110406 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbd6a4-ce87-42bf-8b2d-85f8fd58140a admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:20:54.111048 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbbd6a4-ce87-42bf-8b2d-85f8fd58140a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:20:54.112610 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbbd6a4-ce87-42bf-8b2d-85f8fd58140a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:20:54.122995 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1232/2477] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:20:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:20:54.127286 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe4036-8e37-4abb-818c-46f93109c3d9 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:20:54.137050 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe4036-8e37-4abb-818c-46f93109c3d9 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:20:54.142304 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e3a03-5001-45bd-9d70-cf2845afd5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:20:54.142944 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e3a03-5001-45bd-9d70-cf2845afd5a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:20:54.143226 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e3a03-5001-45bd-9d70-cf2845afd5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:20:54.143498 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e3a03-5001-45bd-9d70-cf2845afd5a6 admin admin] PATH_INFO: `/v3/projects/121323b3ac0648219a90de08f2fd5c03/users/597c6177703a4536adaaaa3ff5a3b28b/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:20:54.157726 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46e3a03-5001-45bd-9d70-cf2845afd5a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=121323b3ac0648219a90de08f2fd5c03, user_id=597c6177703a4536adaaaa3ff5a3b28b, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:20:54.160425 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46e3a03-5001-45bd-9d70-cf2845afd5a6 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:20:54.173330 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1246/2478] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:20:54 2022] PUT /identity/v3/projects/121323b3ac0648219a90de08f2fd5c03/users/597c6177703a4536adaaaa3ff5a3b28b/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:20:54.183219 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd091f-a867-4bd0-9c66-e781a265de62 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:20:54.183502 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd091f-a867-4bd0-9c66-e781a265de62 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:20:54.183774 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dd091f-a867-4bd0-9c66-e781a265de62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:20:54.212696 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-c7dd091f-a867-4bd0-9c66-e781a265de62 None None] MFA Rules not processed for user `597c6177703a4536adaaaa3ff5a3b28b`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:20:54.213909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c7dd091f-a867-4bd0-9c66-e781a265de62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:20:54.276252 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1233/2479] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:20:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:20:54.287090 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e3a03-5001-45bd-9d70-cf2845afd5a6 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:20:54.288850 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-f46e3a03-5001-45bd-9d70-cf2845afd5a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:20:54.344805 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46e3a03-5001-45bd-9d70-cf2845afd5a6 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:20:54.348466 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc34495b-fb3b-4b94-9f29-1c6d66ab7db2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:20:54.349061 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34495b-fb3b-4b94-9f29-1c6d66ab7db2 service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:20:54.349339 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34495b-fb3b-4b94-9f29-1c6d66ab7db2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:20:54.349608 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc34495b-fb3b-4b94-9f29-1c6d66ab7db2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:20:54.357714 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc34495b-fb3b-4b94-9f29-1c6d66ab7db2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:20:54.359187 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc34495b-fb3b-4b94-9f29-1c6d66ab7db2 service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:20:54.368307 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1247/2480] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:20:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:21:02.702281 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd091f-a867-4bd0-9c66-e781a265de62 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:02.765603 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dd091f-a867-4bd0-9c66-e781a265de62 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:02.770015 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a67772-2535-4d1f-a03e-b31a15b51a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:02.770671 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a67772-2535-4d1f-a03e-b31a15b51a94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:02.770948 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a67772-2535-4d1f-a03e-b31a15b51a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:02.771218 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a67772-2535-4d1f-a03e-b31a15b51a94 admin admin] PATH_INFO: `/v3/users/597c6177703a4536adaaaa3ff5a3b28b` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:02.772323 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a67772-2535-4d1f-a03e-b31a15b51a94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=597c6177703a4536adaaaa3ff5a3b28b)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:02.774217 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a67772-2535-4d1f-a03e-b31a15b51a94 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:02.776490 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:21:02.821451 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-75a67772-2535-4d1f-a03e-b31a15b51a94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '597c6177703a4536adaaaa3ff5a3b28b'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:02.833980 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-75a67772-2535-4d1f-a03e-b31a15b51a94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '597c6177703a4536adaaaa3ff5a3b28b'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:02.840424 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-75a67772-2535-4d1f-a03e-b31a15b51a94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '597c6177703a4536adaaaa3ff5a3b28b'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:02.841172 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-75a67772-2535-4d1f-a03e-b31a15b51a94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '597c6177703a4536adaaaa3ff5a3b28b'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:02.848423 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-75a67772-2535-4d1f-a03e-b31a15b51a94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '597c6177703a4536adaaaa3ff5a3b28b'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:02.854405 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1234/2481] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:21:02 2022] DELETE /identity/v3/users/597c6177703a4536adaaaa3ff5a3b28b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:21:02.858826 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee806fde-7165-41d4-9a7d-c09a5f17958e None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:02.859304 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee806fde-7165-41d4-9a7d-c09a5f17958e None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:02.859408 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee806fde-7165-41d4-9a7d-c09a5f17958e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:02.879285 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-ee806fde-7165-41d4-9a7d-c09a5f17958e None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:21:02.880477 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-ee806fde-7165-41d4-9a7d-c09a5f17958e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:02.938530 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1248/2482] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:21:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:21:02.948092 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a67772-2535-4d1f-a03e-b31a15b51a94 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:02.949472 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-75a67772-2535-4d1f-a03e-b31a15b51a94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:03.006077 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a67772-2535-4d1f-a03e-b31a15b51a94 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:03.009662 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c82d0-b770-4f73-9439-c6b5f79dc23c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:03.010273 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c82d0-b770-4f73-9439-c6b5f79dc23c service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:03.010560 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c82d0-b770-4f73-9439-c6b5f79dc23c service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:03.010836 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c82d0-b770-4f73-9439-c6b5f79dc23c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:03.020008 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c82d0-b770-4f73-9439-c6b5f79dc23c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:03.021420 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c82d0-b770-4f73-9439-c6b5f79dc23c service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:03.030854 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1235/2483] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:21:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 20:21:03.291964 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee806fde-7165-41d4-9a7d-c09a5f17958e None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:03.293429 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-ee806fde-7165-41d4-9a7d-c09a5f17958e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:03.305773 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee806fde-7165-41d4-9a7d-c09a5f17958e None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:03.310453 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58b868c-b216-4c1e-80fd-d34bf3306996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:03.311083 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b868c-b216-4c1e-80fd-d34bf3306996 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:03.311360 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b868c-b216-4c1e-80fd-d34bf3306996 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:03.311635 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58b868c-b216-4c1e-80fd-d34bf3306996 admin admin] PATH_INFO: `/v3/projects/121323b3ac0648219a90de08f2fd5c03` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:03.312740 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58b868c-b216-4c1e-80fd-d34bf3306996 admin admin] RBAC: Authorizing `identity:delete_project(project_id=121323b3ac0648219a90de08f2fd5c03)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:03.314660 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58b868c-b216-4c1e-80fd-d34bf3306996 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:03.363836 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c58b868c-b216-4c1e-80fd-d34bf3306996 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '121323b3ac0648219a90de08f2fd5c03', 'request_id': 'req-c58b868c-b216-4c1e-80fd-d34bf3306996'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:03.367407 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:21:03.372165 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c58b868c-b216-4c1e-80fd-d34bf3306996 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '121323b3ac0648219a90de08f2fd5c03', 'request_id': 'req-c58b868c-b216-4c1e-80fd-d34bf3306996'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:03.379300 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c58b868c-b216-4c1e-80fd-d34bf3306996 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '121323b3ac0648219a90de08f2fd5c03', 'request_id': 'req-c58b868c-b216-4c1e-80fd-d34bf3306996'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:03.390424 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c58b868c-b216-4c1e-80fd-d34bf3306996 admin admin] The token cache is being invalidate because project 121323b3ac0648219a90de08f2fd5c03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:21:03.390715 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c58b868c-b216-4c1e-80fd-d34bf3306996 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:03.391851 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c58b868c-b216-4c1e-80fd-d34bf3306996 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:03.393330 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1249/2484] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:21:03 2022] DELETE /identity/v3/projects/121323b3ac0648219a90de08f2fd5c03 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:21:03.398322 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2d219-95e4-4dcb-a4fd-61301cae855b None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:03.398613 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2d219-95e4-4dcb-a4fd-61301cae855b None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:03.398890 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2d219-95e4-4dcb-a4fd-61301cae855b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:03.418632 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-12a2d219-95e4-4dcb-a4fd-61301cae855b None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:21:03.419837 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-12a2d219-95e4-4dcb-a4fd-61301cae855b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:03.479462 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1236/2485] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:21:03.488117 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca72130-39fa-4a4b-b5de-e4cd5e74c44f None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:03.488405 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca72130-39fa-4a4b-b5de-e4cd5e74c44f None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:03.488683 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca72130-39fa-4a4b-b5de-e4cd5e74c44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:03.508048 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-dca72130-39fa-4a4b-b5de-e4cd5e74c44f None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:21:03.509241 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-dca72130-39fa-4a4b-b5de-e4cd5e74c44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:03.517533 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1250/2486] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:21:03.521535 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2d219-95e4-4dcb-a4fd-61301cae855b None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:03.531174 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2d219-95e4-4dcb-a4fd-61301cae855b None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:03.536387 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7768506b-f099-4d97-9d13-d16ba2b74920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:03.537012 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768506b-f099-4d97-9d13-d16ba2b74920 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:03.537292 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768506b-f099-4d97-9d13-d16ba2b74920 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:03.537591 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7768506b-f099-4d97-9d13-d16ba2b74920 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:03.538232 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7768506b-f099-4d97-9d13-d16ba2b74920 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:21:03.538563 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7768506b-f099-4d97-9d13-d16ba2b74920 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:03.540168 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7768506b-f099-4d97-9d13-d16ba2b74920 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:03.551543 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1237/2487] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:21:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:21:03.555556 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca72130-39fa-4a4b-b5de-e4cd5e74c44f None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:03.564849 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca72130-39fa-4a4b-b5de-e4cd5e74c44f None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:03.569560 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185fddd5-7f5f-49da-a491-07603c988005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:03.570160 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fddd5-7f5f-49da-a491-07603c988005 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:03.570476 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fddd5-7f5f-49da-a491-07603c988005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:03.570756 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fddd5-7f5f-49da-a491-07603c988005 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:03.571397 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185fddd5-7f5f-49da-a491-07603c988005 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:03.573289 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185fddd5-7f5f-49da-a491-07603c988005 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:03.600977 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1251/2488] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:21:03 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:21:03.605135 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7768506b-f099-4d97-9d13-d16ba2b74920 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:03.614968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7768506b-f099-4d97-9d13-d16ba2b74920 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:03.622629 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:21:03.673481 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c4727-87c2-4882-a881-eb09476c4c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:03.674078 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c4727-87c2-4882-a881-eb09476c4c19 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:03.674395 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c4727-87c2-4882-a881-eb09476c4c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:03.674670 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c4727-87c2-4882-a881-eb09476c4c19 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:03.675309 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9c4727-87c2-4882-a881-eb09476c4c19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:03.677232 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9c4727-87c2-4882-a881-eb09476c4c19 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:03.698571 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1238/2489] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:21:03 2022] POST /identity/v3/users => generated 285 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:21:03.703019 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185fddd5-7f5f-49da-a491-07603c988005 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:03.713698 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185fddd5-7f5f-49da-a491-07603c988005 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:03.718291 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3af509-ae09-4ea7-ba3d-4f70fc833b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:03.718889 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3af509-ae09-4ea7-ba3d-4f70fc833b78 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:03.719794 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3af509-ae09-4ea7-ba3d-4f70fc833b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:03.720086 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3af509-ae09-4ea7-ba3d-4f70fc833b78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:03.720727 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3af509-ae09-4ea7-ba3d-4f70fc833b78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:03.722345 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3af509-ae09-4ea7-ba3d-4f70fc833b78 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:03.732179 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1252/2490] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:21:03.736293 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c4727-87c2-4882-a881-eb09476c4c19 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:03.746264 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c4727-87c2-4882-a881-eb09476c4c19 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:03.751405 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e97057-eb0c-42ce-932c-296dcafb6464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:03.752019 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e97057-eb0c-42ce-932c-296dcafb6464 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:03.752304 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e97057-eb0c-42ce-932c-296dcafb6464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:03.752579 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e97057-eb0c-42ce-932c-296dcafb6464 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:03.753218 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e97057-eb0c-42ce-932c-296dcafb6464 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:03.754847 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e97057-eb0c-42ce-932c-296dcafb6464 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:03.765262 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1239/2491] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:21:03.769760 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3af509-ae09-4ea7-ba3d-4f70fc833b78 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:03.779625 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3af509-ae09-4ea7-ba3d-4f70fc833b78 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:03.784186 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b66ff-d80a-4b48-9491-2b1a65abd9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:03.784841 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b66ff-d80a-4b48-9491-2b1a65abd9cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:03.787070 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b66ff-d80a-4b48-9491-2b1a65abd9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:03.787374 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b66ff-d80a-4b48-9491-2b1a65abd9cc admin admin] PATH_INFO: `/v3/projects/33e75a84341744f1813f352fcb55b45c/users/616ac3c65e954c5f9fc422a5aaf2cc8a/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:03.802693 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5b66ff-d80a-4b48-9491-2b1a65abd9cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=33e75a84341744f1813f352fcb55b45c, user_id=616ac3c65e954c5f9fc422a5aaf2cc8a, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:03.805379 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5b66ff-d80a-4b48-9491-2b1a65abd9cc admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:03.818163 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1253/2492] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:21:03 2022] PUT /identity/v3/projects/33e75a84341744f1813f352fcb55b45c/users/616ac3c65e954c5f9fc422a5aaf2cc8a/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:21:03.829361 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9a9b3-9920-415a-8362-7cf0c5c29f60 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:03.829649 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9a9b3-9920-415a-8362-7cf0c5c29f60 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:03.829924 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b9a9b3-9920-415a-8362-7cf0c5c29f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:03.858264 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-a8b9a9b3-9920-415a-8362-7cf0c5c29f60 None None] MFA Rules not processed for user `616ac3c65e954c5f9fc422a5aaf2cc8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:21:03.859476 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a8b9a9b3-9920-415a-8362-7cf0c5c29f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:03.923086 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1240/2493] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:21:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:21:03.927565 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b66ff-d80a-4b48-9491-2b1a65abd9cc admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:03.988227 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b66ff-d80a-4b48-9491-2b1a65abd9cc admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:03.992730 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4ddca-dc5e-4382-b85c-cda3d17e942b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:03.993333 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4ddca-dc5e-4382-b85c-cda3d17e942b admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:03.993611 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4ddca-dc5e-4382-b85c-cda3d17e942b admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:03.993882 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de4ddca-dc5e-4382-b85c-cda3d17e942b admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:03.994542 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de4ddca-dc5e-4382-b85c-cda3d17e942b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:03.996423 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de4ddca-dc5e-4382-b85c-cda3d17e942b admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:04.023719 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1254/2494] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:21:03 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:21:04.027763 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b9a9b3-9920-415a-8362-7cf0c5c29f60 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:04.038641 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b9a9b3-9920-415a-8362-7cf0c5c29f60 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:04.095029 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4596a3-c9bc-499e-a274-0b71b5925ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:04.095647 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4596a3-c9bc-499e-a274-0b71b5925ecb admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:04.095929 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4596a3-c9bc-499e-a274-0b71b5925ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:04.096206 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4596a3-c9bc-499e-a274-0b71b5925ecb admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:04.096862 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4596a3-c9bc-499e-a274-0b71b5925ecb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:04.098839 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4596a3-c9bc-499e-a274-0b71b5925ecb admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:04.120149 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1241/2495] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:21:04 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:21:04.124188 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4ddca-dc5e-4382-b85c-cda3d17e942b admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:04.134900 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de4ddca-dc5e-4382-b85c-cda3d17e942b admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:04.139450 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc835eb6-0188-4093-bdd8-035b2a873154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:04.140042 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc835eb6-0188-4093-bdd8-035b2a873154 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:04.140319 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc835eb6-0188-4093-bdd8-035b2a873154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:04.140615 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc835eb6-0188-4093-bdd8-035b2a873154 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:04.141279 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc835eb6-0188-4093-bdd8-035b2a873154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:04.142903 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc835eb6-0188-4093-bdd8-035b2a873154 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:04.153220 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1255/2496] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:21:04.157092 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4596a3-c9bc-499e-a274-0b71b5925ecb admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:04.166804 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4596a3-c9bc-499e-a274-0b71b5925ecb admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:04.171425 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30cfaee-6872-4b76-9699-4c435972cdfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:04.172075 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30cfaee-6872-4b76-9699-4c435972cdfd admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:04.172866 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30cfaee-6872-4b76-9699-4c435972cdfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:04.173141 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30cfaee-6872-4b76-9699-4c435972cdfd admin admin] PATH_INFO: `/v3/projects/16db9c16b239473b9a746b47ffccb143/users/b7eb2cbabc5347c6973fe16be664e510/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:04.176191 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:21:04.189557 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30cfaee-6872-4b76-9699-4c435972cdfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=16db9c16b239473b9a746b47ffccb143, user_id=b7eb2cbabc5347c6973fe16be664e510, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:04.192295 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30cfaee-6872-4b76-9699-4c435972cdfd admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:04.205002 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1242/2497] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:21:04 2022] PUT /identity/v3/projects/16db9c16b239473b9a746b47ffccb143/users/b7eb2cbabc5347c6973fe16be664e510/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:21:04.208886 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc835eb6-0188-4093-bdd8-035b2a873154 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:04.218761 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc835eb6-0188-4093-bdd8-035b2a873154 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:04.266632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:21:04.276493 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53282edf-804f-49fe-aadb-23afe9126d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:04.277096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53282edf-804f-49fe-aadb-23afe9126d37 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:04.277378 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53282edf-804f-49fe-aadb-23afe9126d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:04.277653 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53282edf-804f-49fe-aadb-23afe9126d37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:04.278329 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53282edf-804f-49fe-aadb-23afe9126d37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:04.279915 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53282edf-804f-49fe-aadb-23afe9126d37 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:04.289436 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1256/2498] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:21:04.293846 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30cfaee-6872-4b76-9699-4c435972cdfd admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:04.304822 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30cfaee-6872-4b76-9699-4c435972cdfd admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:04.310073 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b2326-5ed8-49e6-99ba-d57e3f4bbe8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:04.310759 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b2326-5ed8-49e6-99ba-d57e3f4bbe8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:04.311041 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b2326-5ed8-49e6-99ba-d57e3f4bbe8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:04.311315 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951b2326-5ed8-49e6-99ba-d57e3f4bbe8b admin admin] PATH_INFO: `/v3/projects/16db9c16b239473b9a746b47ffccb143/users/b7eb2cbabc5347c6973fe16be664e510/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:04.313837 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b2326-5ed8-49e6-99ba-d57e3f4bbe8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=16db9c16b239473b9a746b47ffccb143, user_id=b7eb2cbabc5347c6973fe16be664e510, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:04.316775 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951b2326-5ed8-49e6-99ba-d57e3f4bbe8b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:04.327123 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1243/2499] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:21:04 2022] PUT /identity/v3/projects/16db9c16b239473b9a746b47ffccb143/users/b7eb2cbabc5347c6973fe16be664e510/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:21:04.337082 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1fdf9d-7335-4edb-9558-7ca6ec71b986 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:04.337373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1fdf9d-7335-4edb-9558-7ca6ec71b986 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:04.337646 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1fdf9d-7335-4edb-9558-7ca6ec71b986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:04.365464 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-5a1fdf9d-7335-4edb-9558-7ca6ec71b986 None None] MFA Rules not processed for user `b7eb2cbabc5347c6973fe16be664e510`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:21:04.366779 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-5a1fdf9d-7335-4edb-9558-7ca6ec71b986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:04.434682 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1257/2500] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:21:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:21:04.446469 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b2326-5ed8-49e6-99ba-d57e3f4bbe8b admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:04.448166 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-951b2326-5ed8-49e6-99ba-d57e3f4bbe8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:04.505233 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951b2326-5ed8-49e6-99ba-d57e3f4bbe8b admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:04.509202 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b685107-dc12-42c8-9ed6-11836bc032ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:04.509796 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b685107-dc12-42c8-9ed6-11836bc032ff service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:04.510075 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b685107-dc12-42c8-9ed6-11836bc032ff service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:04.510376 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b685107-dc12-42c8-9ed6-11836bc032ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:04.519126 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b685107-dc12-42c8-9ed6-11836bc032ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:04.520474 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b685107-dc12-42c8-9ed6-11836bc032ff service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:04.529696 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1244/2501] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:21:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:21:04.606045 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1fdf9d-7335-4edb-9558-7ca6ec71b986 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:04.616381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1fdf9d-7335-4edb-9558-7ca6ec71b986 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:04.620021 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c5fab-e976-4dec-be65-3b9b640ff90d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:04.620619 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c5fab-e976-4dec-be65-3b9b640ff90d service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:04.620901 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c5fab-e976-4dec-be65-3b9b640ff90d service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:04.621175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c5fab-e976-4dec-be65-3b9b640ff90d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:04.677578 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c5fab-e976-4dec-be65-3b9b640ff90d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:04.679032 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c5fab-e976-4dec-be65-3b9b640ff90d service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:04.687867 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1258/2502] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:21:04 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:21:14.525653 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=37,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:21:14.683804 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=67 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:21:36.336261 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b685107-dc12-42c8-9ed6-11836bc032ff service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:36.337674 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-7b685107-dc12-42c8-9ed6-11836bc032ff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:36.397271 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b685107-dc12-42c8-9ed6-11836bc032ff service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:36.400961 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cc4a6e-c97d-4b50-86ab-084d884007b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:36.401557 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc4a6e-c97d-4b50-86ab-084d884007b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:36.401835 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc4a6e-c97d-4b50-86ab-084d884007b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:36.402106 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cc4a6e-c97d-4b50-86ab-084d884007b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:36.403883 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c9cc4a6e-c97d-4b50-86ab-084d884007b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:36.413577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cc4a6e-c97d-4b50-86ab-084d884007b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:36.415027 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cc4a6e-c97d-4b50-86ab-084d884007b1 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:36.423644 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1245/2503] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:21:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:21:46.419461 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=26 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:21:47.184977 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c5fab-e976-4dec-be65-3b9b640ff90d service cinder] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:47.247171 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c5fab-e976-4dec-be65-3b9b640ff90d service cinder] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:47.252183 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d1746-6af2-4641-ab50-17aba3a03603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:47.252799 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d1746-6af2-4641-ab50-17aba3a03603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:47.253076 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d1746-6af2-4641-ab50-17aba3a03603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:47.253345 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0d1746-6af2-4641-ab50-17aba3a03603 admin admin] PATH_INFO: `/v3/users/616ac3c65e954c5f9fc422a5aaf2cc8a` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:47.254470 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d1746-6af2-4641-ab50-17aba3a03603 admin admin] RBAC: Authorizing `identity:delete_user(user_id=616ac3c65e954c5f9fc422a5aaf2cc8a)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:47.256362 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0d1746-6af2-4641-ab50-17aba3a03603 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:47.303732 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ea0d1746-6af2-4641-ab50-17aba3a03603 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '616ac3c65e954c5f9fc422a5aaf2cc8a'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:47.316219 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ea0d1746-6af2-4641-ab50-17aba3a03603 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '616ac3c65e954c5f9fc422a5aaf2cc8a'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:47.323438 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ea0d1746-6af2-4641-ab50-17aba3a03603 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '616ac3c65e954c5f9fc422a5aaf2cc8a'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:47.331580 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ea0d1746-6af2-4641-ab50-17aba3a03603 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '616ac3c65e954c5f9fc422a5aaf2cc8a'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:47.332214 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-ea0d1746-6af2-4641-ab50-17aba3a03603 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '616ac3c65e954c5f9fc422a5aaf2cc8a'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:47.338136 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1259/2504] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:21:47 2022] DELETE /identity/v3/users/616ac3c65e954c5f9fc422a5aaf2cc8a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:21:47.342634 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458053f-a831-4303-8d54-b008800623d0 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:47.342917 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458053f-a831-4303-8d54-b008800623d0 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:47.343194 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458053f-a831-4303-8d54-b008800623d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:47.362320 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-1458053f-a831-4303-8d54-b008800623d0 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:21:47.363502 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-1458053f-a831-4303-8d54-b008800623d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:47.424904 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1246/2505] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:21:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:21:47.434519 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d1746-6af2-4641-ab50-17aba3a03603 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:47.435936 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-ea0d1746-6af2-4641-ab50-17aba3a03603 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:47.493699 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0d1746-6af2-4641-ab50-17aba3a03603 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:47.497310 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd4af6-e9f4-4a05-bfe3-a7b8653180a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:47.497907 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd4af6-e9f4-4a05-bfe3-a7b8653180a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:47.498181 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd4af6-e9f4-4a05-bfe3-a7b8653180a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:47.498478 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfd4af6-e9f4-4a05-bfe3-a7b8653180a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:47.507727 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfd4af6-e9f4-4a05-bfe3-a7b8653180a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:47.509141 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfd4af6-e9f4-4a05-bfe3-a7b8653180a8 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:47.519068 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1260/2506] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:21:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:21:47.787350 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458053f-a831-4303-8d54-b008800623d0 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:47.788903 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-1458053f-a831-4303-8d54-b008800623d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:47.801100 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458053f-a831-4303-8d54-b008800623d0 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:47.806268 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20650428-9343-4228-94bc-ae47cfcd18d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:47.806924 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20650428-9343-4228-94bc-ae47cfcd18d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:47.807207 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20650428-9343-4228-94bc-ae47cfcd18d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:47.807487 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20650428-9343-4228-94bc-ae47cfcd18d7 admin admin] PATH_INFO: `/v3/projects/33e75a84341744f1813f352fcb55b45c` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:47.808650 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20650428-9343-4228-94bc-ae47cfcd18d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33e75a84341744f1813f352fcb55b45c)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:47.810649 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20650428-9343-4228-94bc-ae47cfcd18d7 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:47.860435 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-20650428-9343-4228-94bc-ae47cfcd18d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33e75a84341744f1813f352fcb55b45c', 'request_id': 'req-20650428-9343-4228-94bc-ae47cfcd18d7'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:47.867814 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-20650428-9343-4228-94bc-ae47cfcd18d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33e75a84341744f1813f352fcb55b45c', 'request_id': 'req-20650428-9343-4228-94bc-ae47cfcd18d7'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:47.873276 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-20650428-9343-4228-94bc-ae47cfcd18d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33e75a84341744f1813f352fcb55b45c', 'request_id': 'req-20650428-9343-4228-94bc-ae47cfcd18d7'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:47.884927 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-20650428-9343-4228-94bc-ae47cfcd18d7 admin admin] The token cache is being invalidate because project 33e75a84341744f1813f352fcb55b45c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:21:47.885216 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-20650428-9343-4228-94bc-ae47cfcd18d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:47.885856 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-20650428-9343-4228-94bc-ae47cfcd18d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:47.887385 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1247/2507] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:21:47 2022] DELETE /identity/v3/projects/33e75a84341744f1813f352fcb55b45c => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:21:47.891318 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd4af6-e9f4-4a05-bfe3-a7b8653180a8 service neutron] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:47.901701 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfd4af6-e9f4-4a05-bfe3-a7b8653180a8 service neutron] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:47.919173 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:21:47.958770 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0633a-b03f-4a6e-a18c-547682fffb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:47.959394 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0633a-b03f-4a6e-a18c-547682fffb87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:47.959668 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0633a-b03f-4a6e-a18c-547682fffb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:47.959936 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0633a-b03f-4a6e-a18c-547682fffb87 admin admin] PATH_INFO: `/v3/users/b7eb2cbabc5347c6973fe16be664e510` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:47.961039 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0633a-b03f-4a6e-a18c-547682fffb87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7eb2cbabc5347c6973fe16be664e510)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:47.963013 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0633a-b03f-4a6e-a18c-547682fffb87 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:48.007791 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-23c0633a-b03f-4a6e-a18c-547682fffb87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7eb2cbabc5347c6973fe16be664e510'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:48.024265 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-23c0633a-b03f-4a6e-a18c-547682fffb87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7eb2cbabc5347c6973fe16be664e510'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:48.031176 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-23c0633a-b03f-4a6e-a18c-547682fffb87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7eb2cbabc5347c6973fe16be664e510'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:48.039317 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-23c0633a-b03f-4a6e-a18c-547682fffb87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7eb2cbabc5347c6973fe16be664e510'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:48.039940 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-23c0633a-b03f-4a6e-a18c-547682fffb87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7eb2cbabc5347c6973fe16be664e510'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:48.045921 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1261/2508] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:21:47 2022] DELETE /identity/v3/users/b7eb2cbabc5347c6973fe16be664e510 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:21:48.309937 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20650428-9343-4228-94bc-ae47cfcd18d7 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:48.311427 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-20650428-9343-4228-94bc-ae47cfcd18d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:48.331639 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20650428-9343-4228-94bc-ae47cfcd18d7 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:48.388351 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e832e45c-4e6d-4e87-b611-58720c0d1add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:48.388978 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832e45c-4e6d-4e87-b611-58720c0d1add admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:48.389255 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832e45c-4e6d-4e87-b611-58720c0d1add admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:48.389527 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e832e45c-4e6d-4e87-b611-58720c0d1add admin admin] PATH_INFO: `/v3/projects/16db9c16b239473b9a746b47ffccb143` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:48.390665 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e832e45c-4e6d-4e87-b611-58720c0d1add admin admin] RBAC: Authorizing `identity:delete_project(project_id=16db9c16b239473b9a746b47ffccb143)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:48.393172 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e832e45c-4e6d-4e87-b611-58720c0d1add admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:48.426169 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:21:48.445884 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e832e45c-4e6d-4e87-b611-58720c0d1add admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16db9c16b239473b9a746b47ffccb143', 'request_id': 'req-e832e45c-4e6d-4e87-b611-58720c0d1add'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:48.454597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e832e45c-4e6d-4e87-b611-58720c0d1add admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16db9c16b239473b9a746b47ffccb143', 'request_id': 'req-e832e45c-4e6d-4e87-b611-58720c0d1add'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:48.460411 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e832e45c-4e6d-4e87-b611-58720c0d1add admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16db9c16b239473b9a746b47ffccb143', 'request_id': 'req-e832e45c-4e6d-4e87-b611-58720c0d1add'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:48.472009 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e832e45c-4e6d-4e87-b611-58720c0d1add admin admin] The token cache is being invalidate because project 16db9c16b239473b9a746b47ffccb143 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:21:48.472295 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e832e45c-4e6d-4e87-b611-58720c0d1add admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:48.472952 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-e832e45c-4e6d-4e87-b611-58720c0d1add admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:48.474449 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1248/2509] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:21:48 2022] DELETE /identity/v3/projects/16db9c16b239473b9a746b47ffccb143 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:21:48.479814 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6246018c-5e7e-49ee-8e20-41afca7f14e0 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:48.480292 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6246018c-5e7e-49ee-8e20-41afca7f14e0 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:48.480397 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6246018c-5e7e-49ee-8e20-41afca7f14e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:48.499737 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-6246018c-5e7e-49ee-8e20-41afca7f14e0 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:21:48.500946 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-6246018c-5e7e-49ee-8e20-41afca7f14e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:48.559246 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1262/2510] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:21:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:21:48.566723 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d0e378-8b57-420f-ae2d-e589e769f2f5 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:48.567009 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d0e378-8b57-420f-ae2d-e589e769f2f5 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:48.567284 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d0e378-8b57-420f-ae2d-e589e769f2f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:48.588533 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-c0d0e378-8b57-420f-ae2d-e589e769f2f5 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:21:48.590452 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c0d0e378-8b57-420f-ae2d-e589e769f2f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:48.599018 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1249/2511] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:21:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:21:48.603245 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6246018c-5e7e-49ee-8e20-41afca7f14e0 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:48.612946 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6246018c-5e7e-49ee-8e20-41afca7f14e0 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:48.617507 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afef5928-f93d-47d5-bf64-f702a1539b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:48.618103 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef5928-f93d-47d5-bf64-f702a1539b78 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:48.618423 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef5928-f93d-47d5-bf64-f702a1539b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:48.618703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afef5928-f93d-47d5-bf64-f702a1539b78 admin admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:48.619349 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afef5928-f93d-47d5-bf64-f702a1539b78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:21:48.619658 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afef5928-f93d-47d5-bf64-f702a1539b78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:48.621258 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afef5928-f93d-47d5-bf64-f702a1539b78 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:48.632734 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1263/2512] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:21:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:21:48.636781 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d0e378-8b57-420f-ae2d-e589e769f2f5 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:48.646367 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d0e378-8b57-420f-ae2d-e589e769f2f5 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:48.650828 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2ba441-dce7-4865-aa44-8569f5449123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:48.651411 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ba441-dce7-4865-aa44-8569f5449123 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:48.651684 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ba441-dce7-4865-aa44-8569f5449123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:48.651951 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2ba441-dce7-4865-aa44-8569f5449123 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:48.652571 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2ba441-dce7-4865-aa44-8569f5449123 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:48.654528 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2ba441-dce7-4865-aa44-8569f5449123 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:48.680695 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1250/2513] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:21:48 2022] POST /identity/v3/projects => generated 363 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:21:48.684790 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afef5928-f93d-47d5-bf64-f702a1539b78 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:48.695255 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afef5928-f93d-47d5-bf64-f702a1539b78 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:48.721018 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:21:48.753240 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e79d77-3d3f-4f70-b77a-7f08275aa5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:48.753845 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e79d77-3d3f-4f70-b77a-7f08275aa5f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:48.754123 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e79d77-3d3f-4f70-b77a-7f08275aa5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:48.754432 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e79d77-3d3f-4f70-b77a-7f08275aa5f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:48.755078 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e79d77-3d3f-4f70-b77a-7f08275aa5f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:48.756972 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e79d77-3d3f-4f70-b77a-7f08275aa5f0 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:48.778552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1264/2514] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:21:48 2022] POST /identity/v3/users => generated 282 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:21:48.787939 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2ba441-dce7-4865-aa44-8569f5449123 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:48.798292 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2ba441-dce7-4865-aa44-8569f5449123 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:48.803034 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f005502c-13ec-4530-88b7-0f512112d14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:48.803726 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f005502c-13ec-4530-88b7-0f512112d14a admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:48.804091 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f005502c-13ec-4530-88b7-0f512112d14a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:48.804433 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f005502c-13ec-4530-88b7-0f512112d14a admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:48.805141 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f005502c-13ec-4530-88b7-0f512112d14a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:48.806939 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f005502c-13ec-4530-88b7-0f512112d14a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:48.817807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1251/2515] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:21:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:21:48.821987 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e79d77-3d3f-4f70-b77a-7f08275aa5f0 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:48.832737 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e79d77-3d3f-4f70-b77a-7f08275aa5f0 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:48.837269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdf4998-b0e0-4b5d-86a2-f0c2cc1132ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:48.837859 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf4998-b0e0-4b5d-86a2-f0c2cc1132ae admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:48.838133 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf4998-b0e0-4b5d-86a2-f0c2cc1132ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:48.838431 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdf4998-b0e0-4b5d-86a2-f0c2cc1132ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:48.839065 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf4998-b0e0-4b5d-86a2-f0c2cc1132ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:48.840631 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdf4998-b0e0-4b5d-86a2-f0c2cc1132ae admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:48.852876 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1265/2516] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:21:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:21:48.856998 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f005502c-13ec-4530-88b7-0f512112d14a admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:48.867034 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f005502c-13ec-4530-88b7-0f512112d14a admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:48.871719 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c8b85-1d54-4475-a8c2-cf51c9756bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:48.872371 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c8b85-1d54-4475-a8c2-cf51c9756bba admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:48.872653 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c8b85-1d54-4475-a8c2-cf51c9756bba admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:48.872930 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106c8b85-1d54-4475-a8c2-cf51c9756bba admin admin] PATH_INFO: `/v3/projects/86dd3bf462d145be9f838d78c5db9604/users/c33947f82fe942b985dcce8556ad9852/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:48.887980 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106c8b85-1d54-4475-a8c2-cf51c9756bba admin admin] RBAC: Authorizing `identity:create_grant(project_id=86dd3bf462d145be9f838d78c5db9604, user_id=c33947f82fe942b985dcce8556ad9852, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:48.890735 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106c8b85-1d54-4475-a8c2-cf51c9756bba admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:48.905129 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1252/2517] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:21:48 2022] PUT /identity/v3/projects/86dd3bf462d145be9f838d78c5db9604/users/c33947f82fe942b985dcce8556ad9852/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:21:48.916351 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8111670-a3e3-47d4-bcc1-a14440279b61 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:48.916643 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8111670-a3e3-47d4-bcc1-a14440279b61 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:48.916921 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8111670-a3e3-47d4-bcc1-a14440279b61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:48.945037 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-a8111670-a3e3-47d4-bcc1-a14440279b61 None None] MFA Rules not processed for user `c33947f82fe942b985dcce8556ad9852`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:21:48.946269 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-a8111670-a3e3-47d4-bcc1-a14440279b61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:49.009819 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1266/2518] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:21:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:21:49.013926 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c8b85-1d54-4475-a8c2-cf51c9756bba admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:49.082153 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106c8b85-1d54-4475-a8c2-cf51c9756bba admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:49.085472 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6d8b0-3f81-4f9d-b7bb-cbedec7f1a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:49.086089 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6d8b0-3f81-4f9d-b7bb-cbedec7f1a1f admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:49.086408 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6d8b0-3f81-4f9d-b7bb-cbedec7f1a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:49.086685 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6d8b0-3f81-4f9d-b7bb-cbedec7f1a1f admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:49.087323 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab6d8b0-3f81-4f9d-b7bb-cbedec7f1a1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:49.089221 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab6d8b0-3f81-4f9d-b7bb-cbedec7f1a1f admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:49.116889 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1253/2519] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:21:49 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:21:49.121031 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8111670-a3e3-47d4-bcc1-a14440279b61 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:49.132636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8111670-a3e3-47d4-bcc1-a14440279b61 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:49.192721 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4249f2b0-78e7-4f55-a8cc-ce7d0bc8aa0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:49.193243 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249f2b0-78e7-4f55-a8cc-ce7d0bc8aa0c admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:49.193523 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249f2b0-78e7-4f55-a8cc-ce7d0bc8aa0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:49.193795 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4249f2b0-78e7-4f55-a8cc-ce7d0bc8aa0c admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:49.194454 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4249f2b0-78e7-4f55-a8cc-ce7d0bc8aa0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:49.196361 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4249f2b0-78e7-4f55-a8cc-ce7d0bc8aa0c admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:49.218079 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1267/2520] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:21:49 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:21:49.222556 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6d8b0-3f81-4f9d-b7bb-cbedec7f1a1f admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:49.233080 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6d8b0-3f81-4f9d-b7bb-cbedec7f1a1f admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:49.238491 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ebf0e-68fb-4f39-8bec-4fcf36e774d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:49.239089 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ebf0e-68fb-4f39-8bec-4fcf36e774d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:49.239364 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ebf0e-68fb-4f39-8bec-4fcf36e774d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:49.239632 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023ebf0e-68fb-4f39-8bec-4fcf36e774d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:49.240267 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023ebf0e-68fb-4f39-8bec-4fcf36e774d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:49.241853 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023ebf0e-68fb-4f39-8bec-4fcf36e774d2 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:49.253304 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1254/2521] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:21:49.257557 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4249f2b0-78e7-4f55-a8cc-ce7d0bc8aa0c admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:49.267580 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4249f2b0-78e7-4f55-a8cc-ce7d0bc8aa0c admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:49.272214 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857c811-2d6e-43e8-a656-5fff744a9abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:49.272869 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857c811-2d6e-43e8-a656-5fff744a9abd admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:49.273147 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857c811-2d6e-43e8-a656-5fff744a9abd admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:49.273420 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4857c811-2d6e-43e8-a656-5fff744a9abd admin admin] PATH_INFO: `/v3/projects/326ffffa818042d09fe3b6bf87b19aaa/users/286f1fdfb4eb49ae8bc321b07c924c6b/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:49.288101 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4857c811-2d6e-43e8-a656-5fff744a9abd admin admin] RBAC: Authorizing `identity:create_grant(project_id=326ffffa818042d09fe3b6bf87b19aaa, user_id=286f1fdfb4eb49ae8bc321b07c924c6b, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:49.290980 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4857c811-2d6e-43e8-a656-5fff744a9abd admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:49.304510 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1268/2522] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:21:49 2022] PUT /identity/v3/projects/326ffffa818042d09fe3b6bf87b19aaa/users/286f1fdfb4eb49ae8bc321b07c924c6b/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:21:49.308646 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ebf0e-68fb-4f39-8bec-4fcf36e774d2 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:49.320469 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023ebf0e-68fb-4f39-8bec-4fcf36e774d2 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:49.341326 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:21:49.383086 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9223bbe0-2c6d-4e5e-83c9-cae41a725c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:49.383704 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223bbe0-2c6d-4e5e-83c9-cae41a725c74 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:49.383980 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223bbe0-2c6d-4e5e-83c9-cae41a725c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:49.384256 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9223bbe0-2c6d-4e5e-83c9-cae41a725c74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:49.384894 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9223bbe0-2c6d-4e5e-83c9-cae41a725c74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:49.386514 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9223bbe0-2c6d-4e5e-83c9-cae41a725c74 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:49.396595 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1255/2523] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:21:49 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 20:21:49.400867 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857c811-2d6e-43e8-a656-5fff744a9abd admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:49.406335 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:21:49.414619 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4857c811-2d6e-43e8-a656-5fff744a9abd admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:49.423414 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f598823-5cdd-4292-a659-d9ed7ec504ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:49.424100 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f598823-5cdd-4292-a659-d9ed7ec504ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:49.424439 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f598823-5cdd-4292-a659-d9ed7ec504ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:49.424632 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f598823-5cdd-4292-a659-d9ed7ec504ae admin admin] PATH_INFO: `/v3/projects/326ffffa818042d09fe3b6bf87b19aaa/users/286f1fdfb4eb49ae8bc321b07c924c6b/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:49.426655 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f598823-5cdd-4292-a659-d9ed7ec504ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=326ffffa818042d09fe3b6bf87b19aaa, user_id=286f1fdfb4eb49ae8bc321b07c924c6b, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:49.429399 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f598823-5cdd-4292-a659-d9ed7ec504ae admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:49.440963 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1269/2524] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:21:49 2022] PUT /identity/v3/projects/326ffffa818042d09fe3b6bf87b19aaa/users/286f1fdfb4eb49ae8bc321b07c924c6b/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:21:49.451262 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047c1b0-26c6-4f90-9097-52017dab441e None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:49.451751 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047c1b0-26c6-4f90-9097-52017dab441e None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:49.451751 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c047c1b0-26c6-4f90-9097-52017dab441e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:49.480794 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-c047c1b0-26c6-4f90-9097-52017dab441e None None] MFA Rules not processed for user `286f1fdfb4eb49ae8bc321b07c924c6b`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:21:49.482017 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-c047c1b0-26c6-4f90-9097-52017dab441e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:49.550939 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1256/2525] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:21:49 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:21:49.563878 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f598823-5cdd-4292-a659-d9ed7ec504ae admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:49.565641 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-4f598823-5cdd-4292-a659-d9ed7ec504ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:49.622552 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f598823-5cdd-4292-a659-d9ed7ec504ae admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:49.626303 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ae1b77-9c0e-49b2-9718-118a6dd045eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:49.626908 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae1b77-9c0e-49b2-9718-118a6dd045eb service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:49.627183 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae1b77-9c0e-49b2-9718-118a6dd045eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:49.627451 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ae1b77-9c0e-49b2-9718-118a6dd045eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:49.636385 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae1b77-9c0e-49b2-9718-118a6dd045eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:49.637828 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ae1b77-9c0e-49b2-9718-118a6dd045eb service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:49.647188 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1270/2526] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:21:49 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:21:50.061219 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c047c1b0-26c6-4f90-9097-52017dab441e None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:50.124830 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c047c1b0-26c6-4f90-9097-52017dab441e None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:50.129414 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c40e25-7383-45bc-98cd-d8816aa77fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:50.130047 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c40e25-7383-45bc-98cd-d8816aa77fc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:50.130358 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c40e25-7383-45bc-98cd-d8816aa77fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:50.130638 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c40e25-7383-45bc-98cd-d8816aa77fc2 admin admin] PATH_INFO: `/v3/users/c33947f82fe942b985dcce8556ad9852` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:50.131762 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c40e25-7383-45bc-98cd-d8816aa77fc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c33947f82fe942b985dcce8556ad9852)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:50.133713 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c40e25-7383-45bc-98cd-d8816aa77fc2 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:50.178906 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-53c40e25-7383-45bc-98cd-d8816aa77fc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c33947f82fe942b985dcce8556ad9852'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:50.180025 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:21:50.192990 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-53c40e25-7383-45bc-98cd-d8816aa77fc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c33947f82fe942b985dcce8556ad9852'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:50.200102 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-53c40e25-7383-45bc-98cd-d8816aa77fc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c33947f82fe942b985dcce8556ad9852'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:50.200972 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-53c40e25-7383-45bc-98cd-d8816aa77fc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c33947f82fe942b985dcce8556ad9852'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:50.208541 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-53c40e25-7383-45bc-98cd-d8816aa77fc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c33947f82fe942b985dcce8556ad9852'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:50.214396 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1257/2527] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:21:50 2022] DELETE /identity/v3/users/c33947f82fe942b985dcce8556ad9852 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:21:50.218939 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f642d8c-6165-4c8f-91aa-d69227ede807 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:50.219174 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f642d8c-6165-4c8f-91aa-d69227ede807 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:50.219443 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f642d8c-6165-4c8f-91aa-d69227ede807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:50.238476 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-6f642d8c-6165-4c8f-91aa-d69227ede807 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:21:50.239671 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-6f642d8c-6165-4c8f-91aa-d69227ede807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:50.297119 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1271/2528] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:21:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:21:50.306842 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c40e25-7383-45bc-98cd-d8816aa77fc2 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:50.308221 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-53c40e25-7383-45bc-98cd-d8816aa77fc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:50.365064 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c40e25-7383-45bc-98cd-d8816aa77fc2 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:50.368762 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad56cd3b-c05b-4d64-bc82-668c62276df6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:50.369372 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56cd3b-c05b-4d64-bc82-668c62276df6 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:50.369656 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56cd3b-c05b-4d64-bc82-668c62276df6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:50.369933 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad56cd3b-c05b-4d64-bc82-668c62276df6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:50.379485 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad56cd3b-c05b-4d64-bc82-668c62276df6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:50.380949 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad56cd3b-c05b-4d64-bc82-668c62276df6 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:50.390562 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1258/2529] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:21:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:21:50.650786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f642d8c-6165-4c8f-91aa-d69227ede807 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:50.652360 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-6f642d8c-6165-4c8f-91aa-d69227ede807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:50.664880 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f642d8c-6165-4c8f-91aa-d69227ede807 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:50.669608 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71df7896-7264-473d-89f1-e526bba38cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:50.670286 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df7896-7264-473d-89f1-e526bba38cfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:50.670560 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df7896-7264-473d-89f1-e526bba38cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:50.670836 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df7896-7264-473d-89f1-e526bba38cfa admin admin] PATH_INFO: `/v3/projects/86dd3bf462d145be9f838d78c5db9604` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:50.671976 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71df7896-7264-473d-89f1-e526bba38cfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=86dd3bf462d145be9f838d78c5db9604)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:50.674465 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71df7896-7264-473d-89f1-e526bba38cfa admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:50.721278 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-71df7896-7264-473d-89f1-e526bba38cfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86dd3bf462d145be9f838d78c5db9604', 'request_id': 'req-71df7896-7264-473d-89f1-e526bba38cfa'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:50.728002 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-71df7896-7264-473d-89f1-e526bba38cfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86dd3bf462d145be9f838d78c5db9604', 'request_id': 'req-71df7896-7264-473d-89f1-e526bba38cfa'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:50.733998 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-71df7896-7264-473d-89f1-e526bba38cfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86dd3bf462d145be9f838d78c5db9604', 'request_id': 'req-71df7896-7264-473d-89f1-e526bba38cfa'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:50.745404 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-71df7896-7264-473d-89f1-e526bba38cfa admin admin] The token cache is being invalidate because project 86dd3bf462d145be9f838d78c5db9604 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:21:50.745697 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-71df7896-7264-473d-89f1-e526bba38cfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:50.746361 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-71df7896-7264-473d-89f1-e526bba38cfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:50.747841 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1272/2530] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:21:50 2022] DELETE /identity/v3/projects/86dd3bf462d145be9f838d78c5db9604 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:21:50.752452 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad56cd3b-c05b-4d64-bc82-668c62276df6 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:50.761830 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad56cd3b-c05b-4d64-bc82-668c62276df6 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:50.818223 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f6d987-5017-4723-9a30-c756cbfdce68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:50.818887 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6d987-5017-4723-9a30-c756cbfdce68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:50.819165 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6d987-5017-4723-9a30-c756cbfdce68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:50.819438 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f6d987-5017-4723-9a30-c756cbfdce68 admin admin] PATH_INFO: `/v3/users/286f1fdfb4eb49ae8bc321b07c924c6b` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:50.820561 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6d987-5017-4723-9a30-c756cbfdce68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=286f1fdfb4eb49ae8bc321b07c924c6b)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:50.822532 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f6d987-5017-4723-9a30-c756cbfdce68 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:50.869278 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-b9f6d987-5017-4723-9a30-c756cbfdce68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '286f1fdfb4eb49ae8bc321b07c924c6b'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:50.881743 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-b9f6d987-5017-4723-9a30-c756cbfdce68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '286f1fdfb4eb49ae8bc321b07c924c6b'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:50.885242 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:21:50.890657 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-b9f6d987-5017-4723-9a30-c756cbfdce68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '286f1fdfb4eb49ae8bc321b07c924c6b'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:50.892191 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-b9f6d987-5017-4723-9a30-c756cbfdce68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '286f1fdfb4eb49ae8bc321b07c924c6b'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:50.901661 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-b9f6d987-5017-4723-9a30-c756cbfdce68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '286f1fdfb4eb49ae8bc321b07c924c6b'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:50.907950 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1259/2531] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:21:50 2022] DELETE /identity/v3/users/286f1fdfb4eb49ae8bc321b07c924c6b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:21:51.170236 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71df7896-7264-473d-89f1-e526bba38cfa admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:51.171703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-71df7896-7264-473d-89f1-e526bba38cfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:51.185673 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71df7896-7264-473d-89f1-e526bba38cfa admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:51.198749 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:21:51.249275 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e53488-a52e-41f0-b9c9-b800414930f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:51.249837 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e53488-a52e-41f0-b9c9-b800414930f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:51.250125 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e53488-a52e-41f0-b9c9-b800414930f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:51.250966 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e53488-a52e-41f0-b9c9-b800414930f8 admin admin] PATH_INFO: `/v3/projects/326ffffa818042d09fe3b6bf87b19aaa` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:51.252100 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e53488-a52e-41f0-b9c9-b800414930f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=326ffffa818042d09fe3b6bf87b19aaa)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:51.254041 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e53488-a52e-41f0-b9c9-b800414930f8 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:51.300313 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c3e53488-a52e-41f0-b9c9-b800414930f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '326ffffa818042d09fe3b6bf87b19aaa', 'request_id': 'req-c3e53488-a52e-41f0-b9c9-b800414930f8'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:51.307767 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c3e53488-a52e-41f0-b9c9-b800414930f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '326ffffa818042d09fe3b6bf87b19aaa', 'request_id': 'req-c3e53488-a52e-41f0-b9c9-b800414930f8'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:51.314115 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c3e53488-a52e-41f0-b9c9-b800414930f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '326ffffa818042d09fe3b6bf87b19aaa', 'request_id': 'req-c3e53488-a52e-41f0-b9c9-b800414930f8'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:51.325878 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c3e53488-a52e-41f0-b9c9-b800414930f8 admin admin] The token cache is being invalidate because project 326ffffa818042d09fe3b6bf87b19aaa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:21:51.326173 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c3e53488-a52e-41f0-b9c9-b800414930f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:51.326854 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-c3e53488-a52e-41f0-b9c9-b800414930f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:51.328338 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1273/2532] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:21:51 2022] DELETE /identity/v3/projects/326ffffa818042d09fe3b6bf87b19aaa => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:21:51.333751 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35882dce-3bc7-45f2-8361-394cac9b2de3 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:51.334038 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35882dce-3bc7-45f2-8361-394cac9b2de3 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:51.334358 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35882dce-3bc7-45f2-8361-394cac9b2de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:51.353438 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-35882dce-3bc7-45f2-8361-394cac9b2de3 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:21:51.354668 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-35882dce-3bc7-45f2-8361-394cac9b2de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:51.413105 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1260/2533] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:21:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:21:51.421070 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8d012-a99b-454f-a4c7-b9f5addffa05 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:51.421274 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8d012-a99b-454f-a4c7-b9f5addffa05 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:51.421528 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d8d012-a99b-454f-a4c7-b9f5addffa05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:51.444771 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-d4d8d012-a99b-454f-a4c7-b9f5addffa05 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:21:51.446019 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-d4d8d012-a99b-454f-a4c7-b9f5addffa05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:51.454260 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1274/2534] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:21:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:21:51.458498 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35882dce-3bc7-45f2-8361-394cac9b2de3 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:51.468149 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35882dce-3bc7-45f2-8361-394cac9b2de3 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:51.472768 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb9bc87-7d86-4ca5-a6fb-24d821c89ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:51.473365 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb9bc87-7d86-4ca5-a6fb-24d821c89ee6 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:51.473641 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb9bc87-7d86-4ca5-a6fb-24d821c89ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:51.473911 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb9bc87-7d86-4ca5-a6fb-24d821c89ee6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:51.474608 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb9bc87-7d86-4ca5-a6fb-24d821c89ee6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:21:51.474912 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb9bc87-7d86-4ca5-a6fb-24d821c89ee6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:51.476488 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb9bc87-7d86-4ca5-a6fb-24d821c89ee6 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:51.488455 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1261/2535] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:21:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 20:21:51.492558 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8d012-a99b-454f-a4c7-b9f5addffa05 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:51.502449 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d8d012-a99b-454f-a4c7-b9f5addffa05 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:51.507086 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0e44a-06bb-4957-8254-b849603a79e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:51.507682 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0e44a-06bb-4957-8254-b849603a79e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:51.507991 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0e44a-06bb-4957-8254-b849603a79e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:51.508263 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf0e44a-06bb-4957-8254-b849603a79e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:51.508902 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf0e44a-06bb-4957-8254-b849603a79e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:51.510842 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf0e44a-06bb-4957-8254-b849603a79e4 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:51.536657 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1275/2536] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:21:51 2022] POST /identity/v3/projects => generated 369 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:21:51.540738 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb9bc87-7d86-4ca5-a6fb-24d821c89ee6 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:51.551260 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb9bc87-7d86-4ca5-a6fb-24d821c89ee6 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:51.606577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1adfc-b121-498c-9ac9-b83dfba5e7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:51.607175 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1adfc-b121-498c-9ac9-b83dfba5e7c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:51.607455 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1adfc-b121-498c-9ac9-b83dfba5e7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:51.607730 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b1adfc-b121-498c-9ac9-b83dfba5e7c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:51.608354 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b1adfc-b121-498c-9ac9-b83dfba5e7c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:51.610773 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b1adfc-b121-498c-9ac9-b83dfba5e7c2 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:51.631966 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1262/2537] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:21:51 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:21:51.636043 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0e44a-06bb-4957-8254-b849603a79e4 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:51.646165 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf0e44a-06bb-4957-8254-b849603a79e4 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:51.650718 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49554938-8338-44b2-878a-5b5246120705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:51.651310 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49554938-8338-44b2-878a-5b5246120705 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:51.651886 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49554938-8338-44b2-878a-5b5246120705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:51.651886 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49554938-8338-44b2-878a-5b5246120705 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:51.652465 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49554938-8338-44b2-878a-5b5246120705 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:51.654044 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49554938-8338-44b2-878a-5b5246120705 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:51.664289 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1276/2538] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:21:51.668122 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1adfc-b121-498c-9ac9-b83dfba5e7c2 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:51.679344 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b1adfc-b121-498c-9ac9-b83dfba5e7c2 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:51.683997 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b743984-0023-4462-a2e4-478fe461082f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:51.684590 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b743984-0023-4462-a2e4-478fe461082f admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:51.684871 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b743984-0023-4462-a2e4-478fe461082f admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:51.685146 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b743984-0023-4462-a2e4-478fe461082f admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:51.685809 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b743984-0023-4462-a2e4-478fe461082f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:51.687401 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b743984-0023-4462-a2e4-478fe461082f admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:51.698431 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1263/2539] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:21:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:21:51.702698 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49554938-8338-44b2-878a-5b5246120705 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:51.712539 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49554938-8338-44b2-878a-5b5246120705 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:51.717132 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93696d38-6439-4123-993a-c99bbf643a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:51.717771 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93696d38-6439-4123-993a-c99bbf643a05 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:51.718058 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93696d38-6439-4123-993a-c99bbf643a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:51.718373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93696d38-6439-4123-993a-c99bbf643a05 admin admin] PATH_INFO: `/v3/projects/be8d53e7afe7477e882914341d19840c/users/94079877ebdc41bab481110f3b94c351/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:51.733209 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93696d38-6439-4123-993a-c99bbf643a05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=be8d53e7afe7477e882914341d19840c, user_id=94079877ebdc41bab481110f3b94c351, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:51.737143 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93696d38-6439-4123-993a-c99bbf643a05 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:51.749985 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1277/2540] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:21:51 2022] PUT /identity/v3/projects/be8d53e7afe7477e882914341d19840c/users/94079877ebdc41bab481110f3b94c351/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:21:51.761435 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf5138-a44f-479c-9d7d-a35bf0ad865b None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:51.761923 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf5138-a44f-479c-9d7d-a35bf0ad865b None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:51.761923 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bf5138-a44f-479c-9d7d-a35bf0ad865b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:51.789684 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-13bf5138-a44f-479c-9d7d-a35bf0ad865b None None] MFA Rules not processed for user `94079877ebdc41bab481110f3b94c351`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:21:51.790938 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-13bf5138-a44f-479c-9d7d-a35bf0ad865b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:51.832062 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:21:51.856210 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1264/2541] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:21:51 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:21:51.861218 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93696d38-6439-4123-993a-c99bbf643a05 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:51.904898 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:21:51.923966 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93696d38-6439-4123-993a-c99bbf643a05 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:51.928680 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f2b3e-423a-4aec-8815-27b943127e84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:51.929278 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f2b3e-423a-4aec-8815-27b943127e84 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:51.929561 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f2b3e-423a-4aec-8815-27b943127e84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:51.929836 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9f2b3e-423a-4aec-8815-27b943127e84 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:51.930497 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9f2b3e-423a-4aec-8815-27b943127e84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:51.932419 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9f2b3e-423a-4aec-8815-27b943127e84 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:51.957856 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1278/2542] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:21:51 2022] POST /identity/v3/projects => generated 369 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:21:51.961934 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bf5138-a44f-479c-9d7d-a35bf0ad865b None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:51.972915 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bf5138-a44f-479c-9d7d-a35bf0ad865b None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:52.029808 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c15deb-a339-43f3-adbb-60924a5c9cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:52.030481 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c15deb-a339-43f3-adbb-60924a5c9cc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:52.031159 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c15deb-a339-43f3-adbb-60924a5c9cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:52.031433 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c15deb-a339-43f3-adbb-60924a5c9cc8 admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:52.032588 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c15deb-a339-43f3-adbb-60924a5c9cc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:52.034574 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c15deb-a339-43f3-adbb-60924a5c9cc8 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:52.056170 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1265/2543] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:21:51 2022] POST /identity/v3/users => generated 291 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:21:52.060162 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f2b3e-423a-4aec-8815-27b943127e84 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:52.070465 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9f2b3e-423a-4aec-8815-27b943127e84 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:52.075071 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390305f-4bd0-4f48-ae7a-4d11d0cd89e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:52.075932 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390305f-4bd0-4f48-ae7a-4d11d0cd89e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:52.076213 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390305f-4bd0-4f48-ae7a-4d11d0cd89e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:52.076489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8390305f-4bd0-4f48-ae7a-4d11d0cd89e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:52.077134 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8390305f-4bd0-4f48-ae7a-4d11d0cd89e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:52.079267 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8390305f-4bd0-4f48-ae7a-4d11d0cd89e9 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:52.089828 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1279/2544] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:21:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:21:52.093558 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c15deb-a339-43f3-adbb-60924a5c9cc8 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:52.103912 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c15deb-a339-43f3-adbb-60924a5c9cc8 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:52.108474 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c086a-a836-4bb9-91f5-fab83576c862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:52.109114 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c086a-a836-4bb9-91f5-fab83576c862 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:52.109395 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c086a-a836-4bb9-91f5-fab83576c862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:52.109669 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55c086a-a836-4bb9-91f5-fab83576c862 admin admin] PATH_INFO: `/v3/projects/55f6c34a891f4fd8abe910196ac3a5f1/users/506bb02863eb46799032b4e06e4b1fe1/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:52.124053 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55c086a-a836-4bb9-91f5-fab83576c862 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55f6c34a891f4fd8abe910196ac3a5f1, user_id=506bb02863eb46799032b4e06e4b1fe1, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:52.126816 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55c086a-a836-4bb9-91f5-fab83576c862 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:52.139354 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1266/2545] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:21:52 2022] PUT /identity/v3/projects/55f6c34a891f4fd8abe910196ac3a5f1/users/506bb02863eb46799032b4e06e4b1fe1/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:21:52.143667 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390305f-4bd0-4f48-ae7a-4d11d0cd89e9 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:52.153573 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8390305f-4bd0-4f48-ae7a-4d11d0cd89e9 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:52.208976 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a41e86-be9a-401b-b048-391b841fb7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:52.209714 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a41e86-be9a-401b-b048-391b841fb7e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:52.209994 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a41e86-be9a-401b-b048-391b841fb7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:52.210284 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a41e86-be9a-401b-b048-391b841fb7e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:52.210926 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a41e86-be9a-401b-b048-391b841fb7e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:52.212568 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a41e86-be9a-401b-b048-391b841fb7e8 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:52.222193 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1280/2546] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:21:52 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:21:52.226487 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c086a-a836-4bb9-91f5-fab83576c862 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:52.237661 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55c086a-a836-4bb9-91f5-fab83576c862 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:52.242385 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a9409-7326-4cb9-a27a-3eb91690988d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:52.243034 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a9409-7326-4cb9-a27a-3eb91690988d admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:52.243319 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a9409-7326-4cb9-a27a-3eb91690988d admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:52.243597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8a9409-7326-4cb9-a27a-3eb91690988d admin admin] PATH_INFO: `/v3/projects/55f6c34a891f4fd8abe910196ac3a5f1/users/506bb02863eb46799032b4e06e4b1fe1/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:52.245511 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8a9409-7326-4cb9-a27a-3eb91690988d admin admin] RBAC: Authorizing `identity:create_grant(project_id=55f6c34a891f4fd8abe910196ac3a5f1, user_id=506bb02863eb46799032b4e06e4b1fe1, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:52.248187 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8a9409-7326-4cb9-a27a-3eb91690988d admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:52.258870 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1267/2547] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:21:52 2022] PUT /identity/v3/projects/55f6c34a891f4fd8abe910196ac3a5f1/users/506bb02863eb46799032b4e06e4b1fe1/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:21:52.268886 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6244c32-223c-4df1-ba1e-e67816f62705 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:52.269178 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6244c32-223c-4df1-ba1e-e67816f62705 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:52.269455 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6244c32-223c-4df1-ba1e-e67816f62705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:52.298740 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-f6244c32-223c-4df1-ba1e-e67816f62705 None None] MFA Rules not processed for user `506bb02863eb46799032b4e06e4b1fe1`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:21:52.299895 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-f6244c32-223c-4df1-ba1e-e67816f62705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:52.367480 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1281/2548] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:21:52 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:21:52.378886 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a9409-7326-4cb9-a27a-3eb91690988d admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:52.380595 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-bd8a9409-7326-4cb9-a27a-3eb91690988d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:52.437125 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8a9409-7326-4cb9-a27a-3eb91690988d admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:52.441362 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a7bb6-abee-4fb7-aa09-6d233f1c25c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:52.441961 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a7bb6-abee-4fb7-aa09-6d233f1c25c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:52.442267 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a7bb6-abee-4fb7-aa09-6d233f1c25c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:52.442552 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a7bb6-abee-4fb7-aa09-6d233f1c25c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:52.451718 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a7bb6-abee-4fb7-aa09-6d233f1c25c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:52.453139 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5a7bb6-abee-4fb7-aa09-6d233f1c25c0 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:52.462503 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1268/2549] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:21:52 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:21:53.945706 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6244c32-223c-4df1-ba1e-e67816f62705 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:53.985917 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:21:54.007858 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6244c32-223c-4df1-ba1e-e67816f62705 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:54.012413 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76fa43-2972-4de6-9f89-9c656a434efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:54.013044 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76fa43-2972-4de6-9f89-9c656a434efe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:54.013322 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76fa43-2972-4de6-9f89-9c656a434efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:54.013597 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76fa43-2972-4de6-9f89-9c656a434efe admin admin] PATH_INFO: `/v3/users/94079877ebdc41bab481110f3b94c351` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:54.014751 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e76fa43-2972-4de6-9f89-9c656a434efe admin admin] RBAC: Authorizing `identity:delete_user(user_id=94079877ebdc41bab481110f3b94c351)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:54.017307 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e76fa43-2972-4de6-9f89-9c656a434efe admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:54.061488 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-1e76fa43-2972-4de6-9f89-9c656a434efe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94079877ebdc41bab481110f3b94c351'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:54.073730 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-1e76fa43-2972-4de6-9f89-9c656a434efe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94079877ebdc41bab481110f3b94c351'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:54.081352 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-1e76fa43-2972-4de6-9f89-9c656a434efe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94079877ebdc41bab481110f3b94c351'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:54.088259 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-1e76fa43-2972-4de6-9f89-9c656a434efe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94079877ebdc41bab481110f3b94c351'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:54.088878 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-1e76fa43-2972-4de6-9f89-9c656a434efe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94079877ebdc41bab481110f3b94c351'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:54.094711 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1282/2550] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:21:53 2022] DELETE /identity/v3/users/94079877ebdc41bab481110f3b94c351 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:21:54.099395 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db26357-fb0b-4d62-bd0c-6bf76a35c9d8 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:54.099901 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db26357-fb0b-4d62-bd0c-6bf76a35c9d8 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:54.100018 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db26357-fb0b-4d62-bd0c-6bf76a35c9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:54.118913 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-6db26357-fb0b-4d62-bd0c-6bf76a35c9d8 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:21:54.120091 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-6db26357-fb0b-4d62-bd0c-6bf76a35c9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:54.153584 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:21:54.180525 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1269/2551] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:21:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:21:54.190554 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76fa43-2972-4de6-9f89-9c656a434efe admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:54.192024 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-1e76fa43-2972-4de6-9f89-9c656a434efe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:54.248206 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76fa43-2972-4de6-9f89-9c656a434efe admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:54.252339 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e340f-5eef-49e3-af34-0cedcbdc54d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:54.252938 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e340f-5eef-49e3-af34-0cedcbdc54d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:54.253216 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e340f-5eef-49e3-af34-0cedcbdc54d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:54.253483 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0e340f-5eef-49e3-af34-0cedcbdc54d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:54.262233 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0e340f-5eef-49e3-af34-0cedcbdc54d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:54.263668 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0e340f-5eef-49e3-af34-0cedcbdc54d1 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:54.273591 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1283/2552] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:21:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:21:54.545863 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db26357-fb0b-4d62-bd0c-6bf76a35c9d8 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:54.547821 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-6db26357-fb0b-4d62-bd0c-6bf76a35c9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:54.560054 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db26357-fb0b-4d62-bd0c-6bf76a35c9d8 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:54.565408 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb15bd9-fe86-4ac9-a45f-f48a4e8cb37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:54.566045 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb15bd9-fe86-4ac9-a45f-f48a4e8cb37d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:54.566355 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb15bd9-fe86-4ac9-a45f-f48a4e8cb37d admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:54.566624 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb15bd9-fe86-4ac9-a45f-f48a4e8cb37d admin admin] PATH_INFO: `/v3/projects/be8d53e7afe7477e882914341d19840c` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:54.567747 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb15bd9-fe86-4ac9-a45f-f48a4e8cb37d admin admin] RBAC: Authorizing `identity:delete_project(project_id=be8d53e7afe7477e882914341d19840c)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:54.569642 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb15bd9-fe86-4ac9-a45f-f48a4e8cb37d admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:54.615881 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5eb15bd9-fe86-4ac9-a45f-f48a4e8cb37d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be8d53e7afe7477e882914341d19840c', 'request_id': 'req-5eb15bd9-fe86-4ac9-a45f-f48a4e8cb37d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:54.623101 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5eb15bd9-fe86-4ac9-a45f-f48a4e8cb37d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be8d53e7afe7477e882914341d19840c', 'request_id': 'req-5eb15bd9-fe86-4ac9-a45f-f48a4e8cb37d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:54.629483 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5eb15bd9-fe86-4ac9-a45f-f48a4e8cb37d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be8d53e7afe7477e882914341d19840c', 'request_id': 'req-5eb15bd9-fe86-4ac9-a45f-f48a4e8cb37d'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:54.640487 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5eb15bd9-fe86-4ac9-a45f-f48a4e8cb37d admin admin] The token cache is being invalidate because project be8d53e7afe7477e882914341d19840c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:21:54.640773 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5eb15bd9-fe86-4ac9-a45f-f48a4e8cb37d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:54.641418 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5eb15bd9-fe86-4ac9-a45f-f48a4e8cb37d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:54.642934 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1270/2553] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:21:54 2022] DELETE /identity/v3/projects/be8d53e7afe7477e882914341d19840c => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:21:54.646841 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e340f-5eef-49e3-af34-0cedcbdc54d1 service neutron] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:54.656233 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0e340f-5eef-49e3-af34-0cedcbdc54d1 service neutron] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:54.711182 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88af807b-19d3-4aac-8826-87f695ee4a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:54.711798 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af807b-19d3-4aac-8826-87f695ee4a9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:54.712073 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af807b-19d3-4aac-8826-87f695ee4a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:54.712341 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af807b-19d3-4aac-8826-87f695ee4a9d admin admin] PATH_INFO: `/v3/users/506bb02863eb46799032b4e06e4b1fe1` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:54.713436 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88af807b-19d3-4aac-8826-87f695ee4a9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=506bb02863eb46799032b4e06e4b1fe1)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:54.715362 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88af807b-19d3-4aac-8826-87f695ee4a9d admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:54.716782 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:21:54.765048 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-88af807b-19d3-4aac-8826-87f695ee4a9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '506bb02863eb46799032b4e06e4b1fe1'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:54.777084 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-88af807b-19d3-4aac-8826-87f695ee4a9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '506bb02863eb46799032b4e06e4b1fe1'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:54.783836 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-88af807b-19d3-4aac-8826-87f695ee4a9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '506bb02863eb46799032b4e06e4b1fe1'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:54.790612 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-88af807b-19d3-4aac-8826-87f695ee4a9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '506bb02863eb46799032b4e06e4b1fe1'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:54.791240 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-88af807b-19d3-4aac-8826-87f695ee4a9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '506bb02863eb46799032b4e06e4b1fe1'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:54.797225 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1284/2554] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:21:54 2022] DELETE /identity/v3/users/506bb02863eb46799032b4e06e4b1fe1 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:21:55.061893 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb15bd9-fe86-4ac9-a45f-f48a4e8cb37d admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:55.063993 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5eb15bd9-fe86-4ac9-a45f-f48a4e8cb37d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:55.080283 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb15bd9-fe86-4ac9-a45f-f48a4e8cb37d admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:55.138778 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f316a-66b5-4926-82a6-758657a54403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:55.139436 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f316a-66b5-4926-82a6-758657a54403 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:55.139655 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f316a-66b5-4926-82a6-758657a54403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:55.139926 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f316a-66b5-4926-82a6-758657a54403 admin admin] PATH_INFO: `/v3/projects/55f6c34a891f4fd8abe910196ac3a5f1` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:55.141028 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f316a-66b5-4926-82a6-758657a54403 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55f6c34a891f4fd8abe910196ac3a5f1)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:55.142986 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f316a-66b5-4926-82a6-758657a54403 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:55.187280 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f82f316a-66b5-4926-82a6-758657a54403 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55f6c34a891f4fd8abe910196ac3a5f1', 'request_id': 'req-f82f316a-66b5-4926-82a6-758657a54403'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:55.194946 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f82f316a-66b5-4926-82a6-758657a54403 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55f6c34a891f4fd8abe910196ac3a5f1', 'request_id': 'req-f82f316a-66b5-4926-82a6-758657a54403'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:55.199615 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:21:55.201148 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f82f316a-66b5-4926-82a6-758657a54403 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55f6c34a891f4fd8abe910196ac3a5f1', 'request_id': 'req-f82f316a-66b5-4926-82a6-758657a54403'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:55.216996 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f82f316a-66b5-4926-82a6-758657a54403 admin admin] The token cache is being invalidate because project 55f6c34a891f4fd8abe910196ac3a5f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:21:55.217288 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f82f316a-66b5-4926-82a6-758657a54403 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:55.217941 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f82f316a-66b5-4926-82a6-758657a54403 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:21:55.219449 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1271/2555] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:21:55 2022] DELETE /identity/v3/projects/55f6c34a891f4fd8abe910196ac3a5f1 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:21:55.224847 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00f113-2ef0-477c-8fb2-641f8e2668c1 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:55.225318 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00f113-2ef0-477c-8fb2-641f8e2668c1 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:55.225472 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00f113-2ef0-477c-8fb2-641f8e2668c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:55.245536 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-4c00f113-2ef0-477c-8fb2-641f8e2668c1 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:21:55.246867 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-4c00f113-2ef0-477c-8fb2-641f8e2668c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:55.304108 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1285/2556] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:21:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:21:55.312677 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5c3ed-33d5-4b9d-b14f-1bae5ee5af8e None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:55.313630 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5c3ed-33d5-4b9d-b14f-1bae5ee5af8e None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:55.313911 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5c3ed-33d5-4b9d-b14f-1bae5ee5af8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:55.333909 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-5ab5c3ed-33d5-4b9d-b14f-1bae5ee5af8e None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:21:55.335400 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5ab5c3ed-33d5-4b9d-b14f-1bae5ee5af8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:55.343231 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1272/2557] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:21:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 20:21:55.347375 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00f113-2ef0-477c-8fb2-641f8e2668c1 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:55.356868 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00f113-2ef0-477c-8fb2-641f8e2668c1 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:55.361891 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa8cec-1c5c-4c30-9c76-96f67417e177 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:55.362550 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa8cec-1c5c-4c30-9c76-96f67417e177 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:55.362828 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa8cec-1c5c-4c30-9c76-96f67417e177 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:55.363096 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fa8cec-1c5c-4c30-9c76-96f67417e177 admin admin] PATH_INFO: `/v3/domains` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:55.363733 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa8cec-1c5c-4c30-9c76-96f67417e177 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81412) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:21:55.364028 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa8cec-1c5c-4c30-9c76-96f67417e177 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:55.365596 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1fa8cec-1c5c-4c30-9c76-96f67417e177 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:55.377464 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1286/2558] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:21:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:21:55.381464 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5c3ed-33d5-4b9d-b14f-1bae5ee5af8e None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:55.390505 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5c3ed-33d5-4b9d-b14f-1bae5ee5af8e None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:55.395097 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67512c-4985-49cc-8988-21f15b864876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:55.395690 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67512c-4985-49cc-8988-21f15b864876 admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:55.395968 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67512c-4985-49cc-8988-21f15b864876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:55.396241 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be67512c-4985-49cc-8988-21f15b864876 admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:55.396867 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67512c-4985-49cc-8988-21f15b864876 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:55.398790 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be67512c-4985-49cc-8988-21f15b864876 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:55.425814 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1273/2559] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:21:55 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:21:55.430061 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa8cec-1c5c-4c30-9c76-96f67417e177 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:55.440392 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fa8cec-1c5c-4c30-9c76-96f67417e177 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:55.496485 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1f2a6a-c1c1-45fc-a9c5-b60fee1107c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:55.497090 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f2a6a-c1c1-45fc-a9c5-b60fee1107c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:55.497369 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f2a6a-c1c1-45fc-a9c5-b60fee1107c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:55.497639 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1f2a6a-c1c1-45fc-a9c5-b60fee1107c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:55.498309 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1f2a6a-c1c1-45fc-a9c5-b60fee1107c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:55.500296 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1f2a6a-c1c1-45fc-a9c5-b60fee1107c5 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:55.511058 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:21:55.522703 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1287/2560] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:21:55 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:21:55.526839 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67512c-4985-49cc-8988-21f15b864876 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:55.537600 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be67512c-4985-49cc-8988-21f15b864876 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:55.542469 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7875ce6-8c96-44e5-ba31-7d3e5e979327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:55.543075 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7875ce6-8c96-44e5-ba31-7d3e5e979327 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:55.543357 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7875ce6-8c96-44e5-ba31-7d3e5e979327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:55.543630 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7875ce6-8c96-44e5-ba31-7d3e5e979327 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:55.544269 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7875ce6-8c96-44e5-ba31-7d3e5e979327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:55.545845 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7875ce6-8c96-44e5-ba31-7d3e5e979327 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:55.556997 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1274/2561] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:21:55.560693 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1f2a6a-c1c1-45fc-a9c5-b60fee1107c5 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:55.570579 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1f2a6a-c1c1-45fc-a9c5-b60fee1107c5 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:55.575116 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508dc07f-c769-4702-8043-ae1d2c398904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:55.575998 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508dc07f-c769-4702-8043-ae1d2c398904 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:55.576328 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508dc07f-c769-4702-8043-ae1d2c398904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:55.576713 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508dc07f-c769-4702-8043-ae1d2c398904 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:55.577351 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508dc07f-c769-4702-8043-ae1d2c398904 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:55.578971 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508dc07f-c769-4702-8043-ae1d2c398904 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:55.589573 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1288/2562] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:21:55.593708 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7875ce6-8c96-44e5-ba31-7d3e5e979327 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:55.603964 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7875ce6-8c96-44e5-ba31-7d3e5e979327 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:55.608541 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dbf9cf-8f36-4b55-a99a-cebc325125ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:55.609179 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbf9cf-8f36-4b55-a99a-cebc325125ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:55.609455 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbf9cf-8f36-4b55-a99a-cebc325125ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:55.610369 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbf9cf-8f36-4b55-a99a-cebc325125ff admin admin] PATH_INFO: `/v3/projects/c4e119d26eea436c9087e4eec13b221c/users/f7036a25749e46c78232028ba75c82a7/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:55.624403 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dbf9cf-8f36-4b55-a99a-cebc325125ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4e119d26eea436c9087e4eec13b221c, user_id=f7036a25749e46c78232028ba75c82a7, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:55.627092 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dbf9cf-8f36-4b55-a99a-cebc325125ff admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:55.639529 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1275/2563] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:21:55 2022] PUT /identity/v3/projects/c4e119d26eea436c9087e4eec13b221c/users/f7036a25749e46c78232028ba75c82a7/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:21:55.649607 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4e02b-a510-4f3a-a1b8-a03003a51455 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:55.649892 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4e02b-a510-4f3a-a1b8-a03003a51455 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:55.650163 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d4e02b-a510-4f3a-a1b8-a03003a51455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:55.677670 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-a4d4e02b-a510-4f3a-a1b8-a03003a51455 None None] MFA Rules not processed for user `f7036a25749e46c78232028ba75c82a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:21:55.678886 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-a4d4e02b-a510-4f3a-a1b8-a03003a51455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:55.740716 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1289/2564] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:21:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:21:55.745505 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dbf9cf-8f36-4b55-a99a-cebc325125ff admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:55.806013 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dbf9cf-8f36-4b55-a99a-cebc325125ff admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:55.810688 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5909c14-545f-4bf4-a289-799edd67ee8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:55.811229 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5909c14-545f-4bf4-a289-799edd67ee8d admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:55.811511 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5909c14-545f-4bf4-a289-799edd67ee8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:55.811787 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5909c14-545f-4bf4-a289-799edd67ee8d admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:55.812408 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5909c14-545f-4bf4-a289-799edd67ee8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:55.814371 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5909c14-545f-4bf4-a289-799edd67ee8d admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:55.840153 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1276/2565] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:21:55 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:21:55.844377 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d4e02b-a510-4f3a-a1b8-a03003a51455 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:55.855005 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d4e02b-a510-4f3a-a1b8-a03003a51455 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:55.912192 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac904418-d0f6-4bd4-8a80-dbf46b18ac27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:55.912798 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac904418-d0f6-4bd4-8a80-dbf46b18ac27 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:55.913075 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac904418-d0f6-4bd4-8a80-dbf46b18ac27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:55.913345 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac904418-d0f6-4bd4-8a80-dbf46b18ac27 admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:55.913987 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac904418-d0f6-4bd4-8a80-dbf46b18ac27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:55.915919 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac904418-d0f6-4bd4-8a80-dbf46b18ac27 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:55.940054 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1290/2566] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:21:55 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:21:55.941311 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5909c14-545f-4bf4-a289-799edd67ee8d admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:55.951796 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5909c14-545f-4bf4-a289-799edd67ee8d admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:55.956468 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc979d1-87d6-4a7e-82e4-c2187c82be5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:55.957063 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc979d1-87d6-4a7e-82e4-c2187c82be5a admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:55.957340 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc979d1-87d6-4a7e-82e4-c2187c82be5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:55.957612 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc979d1-87d6-4a7e-82e4-c2187c82be5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:55.958272 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc979d1-87d6-4a7e-82e4-c2187c82be5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:55.959844 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc979d1-87d6-4a7e-82e4-c2187c82be5a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:55.970183 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1277/2567] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:21:55.973966 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac904418-d0f6-4bd4-8a80-dbf46b18ac27 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:55.983801 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac904418-d0f6-4bd4-8a80-dbf46b18ac27 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:55.988894 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c819a202-7f2c-4640-9548-57a507a5c8ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:55.989483 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819a202-7f2c-4640-9548-57a507a5c8ff admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:55.989764 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819a202-7f2c-4640-9548-57a507a5c8ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:55.990038 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c819a202-7f2c-4640-9548-57a507a5c8ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:55.990719 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c819a202-7f2c-4640-9548-57a507a5c8ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:55.992297 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c819a202-7f2c-4640-9548-57a507a5c8ff admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:56.002580 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1291/2568] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:21:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:21:56.006381 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc979d1-87d6-4a7e-82e4-c2187c82be5a admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:56.016187 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc979d1-87d6-4a7e-82e4-c2187c82be5a admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:56.020836 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7982fcd-d088-4d3c-a777-67007bba29be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:56.021489 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7982fcd-d088-4d3c-a777-67007bba29be admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:56.021771 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7982fcd-d088-4d3c-a777-67007bba29be admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:56.022046 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7982fcd-d088-4d3c-a777-67007bba29be admin admin] PATH_INFO: `/v3/projects/78a4bf75d8f34ed0a8ceb4616a351a66/users/d23af9646efa478b951dc0081f99d539/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:56.036588 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7982fcd-d088-4d3c-a777-67007bba29be admin admin] RBAC: Authorizing `identity:create_grant(project_id=78a4bf75d8f34ed0a8ceb4616a351a66, user_id=d23af9646efa478b951dc0081f99d539, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:56.039283 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7982fcd-d088-4d3c-a777-67007bba29be admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:56.052197 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1278/2569] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:21:56 2022] PUT /identity/v3/projects/78a4bf75d8f34ed0a8ceb4616a351a66/users/d23af9646efa478b951dc0081f99d539/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:21:56.062234 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53bbafc-cdfc-4ee7-adf3-5b2985357fd6 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:56.062493 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53bbafc-cdfc-4ee7-adf3-5b2985357fd6 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:56.062790 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53bbafc-cdfc-4ee7-adf3-5b2985357fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:56.090633 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-b53bbafc-cdfc-4ee7-adf3-5b2985357fd6 None None] MFA Rules not processed for user `d23af9646efa478b951dc0081f99d539`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:21:56.091831 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-b53bbafc-cdfc-4ee7-adf3-5b2985357fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:56.132652 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:21:56.156567 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1292/2570] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:21:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:21:56.160973 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7982fcd-d088-4d3c-a777-67007bba29be admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:56.202108 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:21:56.223706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7982fcd-d088-4d3c-a777-67007bba29be admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:56.228383 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909e05d-d329-425b-8c7d-c01152d51bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:56.228970 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909e05d-d329-425b-8c7d-c01152d51bbe admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:56.229250 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909e05d-d329-425b-8c7d-c01152d51bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:56.229521 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6909e05d-d329-425b-8c7d-c01152d51bbe admin admin] PATH_INFO: `/v3/projects` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:56.230161 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6909e05d-d329-425b-8c7d-c01152d51bbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:56.232092 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6909e05d-d329-425b-8c7d-c01152d51bbe admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:56.258408 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1279/2571] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:21:56 2022] POST /identity/v3/projects => generated 387 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:21:56.262466 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53bbafc-cdfc-4ee7-adf3-5b2985357fd6 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:56.272991 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53bbafc-cdfc-4ee7-adf3-5b2985357fd6 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:56.327751 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237118c3-1df8-4747-8ec0-9f0c988f1f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:56.328347 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237118c3-1df8-4747-8ec0-9f0c988f1f73 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:56.328731 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237118c3-1df8-4747-8ec0-9f0c988f1f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:56.329006 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237118c3-1df8-4747-8ec0-9f0c988f1f73 admin admin] PATH_INFO: `/v3/users` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:56.329639 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237118c3-1df8-4747-8ec0-9f0c988f1f73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:56.331556 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237118c3-1df8-4747-8ec0-9f0c988f1f73 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:56.352685 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1293/2572] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:21:56 2022] POST /identity/v3/users => generated 300 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:21:56.356417 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909e05d-d329-425b-8c7d-c01152d51bbe admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:56.367066 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6909e05d-d329-425b-8c7d-c01152d51bbe admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:56.371894 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b66a5-96aa-465d-a917-b4703f4de7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:56.372481 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b66a5-96aa-465d-a917-b4703f4de7c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:56.372759 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b66a5-96aa-465d-a917-b4703f4de7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:56.373028 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8b66a5-96aa-465d-a917-b4703f4de7c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:56.373682 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b66a5-96aa-465d-a917-b4703f4de7c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:56.375501 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8b66a5-96aa-465d-a917-b4703f4de7c9 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:56.385836 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1280/2573] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:21:56.389714 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237118c3-1df8-4747-8ec0-9f0c988f1f73 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:56.399778 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237118c3-1df8-4747-8ec0-9f0c988f1f73 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:56.404378 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcdc989-ac7f-4e46-89f6-33a6c2668dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:56.405020 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcdc989-ac7f-4e46-89f6-33a6c2668dce admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:56.405321 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcdc989-ac7f-4e46-89f6-33a6c2668dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:56.405596 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcdc989-ac7f-4e46-89f6-33a6c2668dce admin admin] PATH_INFO: `/v3/projects/59da7d8bd1df429588d9dd18743e01eb/users/eb3e4460a587473c88b388a3150f6849/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:56.420579 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcdc989-ac7f-4e46-89f6-33a6c2668dce admin admin] RBAC: Authorizing `identity:create_grant(project_id=59da7d8bd1df429588d9dd18743e01eb, user_id=eb3e4460a587473c88b388a3150f6849, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:56.423892 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcdc989-ac7f-4e46-89f6-33a6c2668dce admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:56.436465 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1294/2574] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:21:56 2022] PUT /identity/v3/projects/59da7d8bd1df429588d9dd18743e01eb/users/eb3e4460a587473c88b388a3150f6849/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:21:56.440304 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b66a5-96aa-465d-a917-b4703f4de7c9 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:56.450356 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8b66a5-96aa-465d-a917-b4703f4de7c9 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:56.506999 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944fe75a-4054-485a-9095-d130fe0779ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:56.507597 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944fe75a-4054-485a-9095-d130fe0779ca admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:56.507875 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944fe75a-4054-485a-9095-d130fe0779ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:56.508148 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944fe75a-4054-485a-9095-d130fe0779ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:56.508782 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944fe75a-4054-485a-9095-d130fe0779ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:56.510396 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944fe75a-4054-485a-9095-d130fe0779ca admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:56.519895 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1281/2575] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:21:56 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:21:56.524052 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcdc989-ac7f-4e46-89f6-33a6c2668dce admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:56.534683 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcdc989-ac7f-4e46-89f6-33a6c2668dce admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:56.539263 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e3b9e-2fa7-41cd-8d84-f03c42f3dcf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:56.539911 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e3b9e-2fa7-41cd-8d84-f03c42f3dcf0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:56.540189 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e3b9e-2fa7-41cd-8d84-f03c42f3dcf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:56.540463 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8e3b9e-2fa7-41cd-8d84-f03c42f3dcf0 admin admin] PATH_INFO: `/v3/projects/59da7d8bd1df429588d9dd18743e01eb/users/eb3e4460a587473c88b388a3150f6849/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:56.542419 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8e3b9e-2fa7-41cd-8d84-f03c42f3dcf0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59da7d8bd1df429588d9dd18743e01eb, user_id=eb3e4460a587473c88b388a3150f6849, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:56.545050 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8e3b9e-2fa7-41cd-8d84-f03c42f3dcf0 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:56.555839 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1295/2576] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:21:56 2022] PUT /identity/v3/projects/59da7d8bd1df429588d9dd18743e01eb/users/eb3e4460a587473c88b388a3150f6849/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:21:56.566671 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e3cbd-d1dc-4505-b72e-78f515dc8087 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:56.566954 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e3cbd-d1dc-4505-b72e-78f515dc8087 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:56.567227 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03e3cbd-d1dc-4505-b72e-78f515dc8087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:56.595677 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-b03e3cbd-d1dc-4505-b72e-78f515dc8087 None None] MFA Rules not processed for user `eb3e4460a587473c88b388a3150f6849`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:21:56.596805 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-b03e3cbd-d1dc-4505-b72e-78f515dc8087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:56.663244 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1282/2577] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:21:56 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:21:56.674078 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e3b9e-2fa7-41cd-8d84-f03c42f3dcf0 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:56.675810 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-cf8e3b9e-2fa7-41cd-8d84-f03c42f3dcf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:56.734394 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8e3b9e-2fa7-41cd-8d84-f03c42f3dcf0 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:56.738051 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb724f8c-e706-4d11-add8-49de1e23954a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:56.738682 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb724f8c-e706-4d11-add8-49de1e23954a service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:56.738966 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb724f8c-e706-4d11-add8-49de1e23954a service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:56.739239 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb724f8c-e706-4d11-add8-49de1e23954a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:56.748115 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb724f8c-e706-4d11-add8-49de1e23954a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:56.749555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb724f8c-e706-4d11-add8-49de1e23954a service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:56.758880 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1296/2578] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:21:56 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:21:57.255982 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e3cbd-d1dc-4505-b72e-78f515dc8087 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:21:57.257451 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-b03e3cbd-d1dc-4505-b72e-78f515dc8087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:21:57.270135 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03e3cbd-d1dc-4505-b72e-78f515dc8087 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:21:57.273865 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e191d4-bab8-4428-8bff-81c6e89fc6af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:21:57.274498 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e191d4-bab8-4428-8bff-81c6e89fc6af service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:21:57.274773 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e191d4-bab8-4428-8bff-81c6e89fc6af service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:21:57.275047 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e191d4-bab8-4428-8bff-81c6e89fc6af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:21:57.306781 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:21:57.338233 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e191d4-bab8-4428-8bff-81c6e89fc6af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:21:57.339689 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e191d4-bab8-4428-8bff-81c6e89fc6af service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:21:57.349017 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1283/2579] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:21:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:22:01.363371 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb724f8c-e706-4d11-add8-49de1e23954a service cinder] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:22:01.372131 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb724f8c-e706-4d11-add8-49de1e23954a service cinder] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:22:01.375777 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64afca3d-cdcb-493f-b1fd-4d6392fc126c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:22:01.376373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64afca3d-cdcb-493f-b1fd-4d6392fc126c service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:01.376645 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64afca3d-cdcb-493f-b1fd-4d6392fc126c service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:01.377526 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64afca3d-cdcb-493f-b1fd-4d6392fc126c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:01.410567 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:22:01.434941 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64afca3d-cdcb-493f-b1fd-4d6392fc126c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:22:01.436381 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64afca3d-cdcb-493f-b1fd-4d6392fc126c service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:22:01.445431 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1297/2580] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:22:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:22:03.490733 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e191d4-bab8-4428-8bff-81c6e89fc6af service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:22:03.552152 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e191d4-bab8-4428-8bff-81c6e89fc6af service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:22:03.556715 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50195398-dfd7-4bde-aa2f-a6dd90c6586a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:22:03.557275 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50195398-dfd7-4bde-aa2f-a6dd90c6586a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:03.557553 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50195398-dfd7-4bde-aa2f-a6dd90c6586a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:03.557825 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50195398-dfd7-4bde-aa2f-a6dd90c6586a admin admin] PATH_INFO: `/v3/users/f7036a25749e46c78232028ba75c82a7` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:03.558948 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50195398-dfd7-4bde-aa2f-a6dd90c6586a admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7036a25749e46c78232028ba75c82a7)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:22:03.560930 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50195398-dfd7-4bde-aa2f-a6dd90c6586a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:22:03.606885 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-50195398-dfd7-4bde-aa2f-a6dd90c6586a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7036a25749e46c78232028ba75c82a7'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:03.619186 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-50195398-dfd7-4bde-aa2f-a6dd90c6586a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7036a25749e46c78232028ba75c82a7'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:03.627620 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-50195398-dfd7-4bde-aa2f-a6dd90c6586a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7036a25749e46c78232028ba75c82a7'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:03.628245 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-50195398-dfd7-4bde-aa2f-a6dd90c6586a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7036a25749e46c78232028ba75c82a7'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:03.635328 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-50195398-dfd7-4bde-aa2f-a6dd90c6586a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7036a25749e46c78232028ba75c82a7'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:03.641429 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1284/2581] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:22:03 2022] DELETE /identity/v3/users/f7036a25749e46c78232028ba75c82a7 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:22:03.645949 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c611f2-feab-45cd-a2ae-11ed945fae4e None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:03.646456 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c611f2-feab-45cd-a2ae-11ed945fae4e None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:03.646564 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c611f2-feab-45cd-a2ae-11ed945fae4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:03.665702 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-c9c611f2-feab-45cd-a2ae-11ed945fae4e None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:22:03.666941 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-c9c611f2-feab-45cd-a2ae-11ed945fae4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:22:03.724043 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1298/2582] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:22:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:22:03.733577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50195398-dfd7-4bde-aa2f-a6dd90c6586a admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:22:03.735038 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-50195398-dfd7-4bde-aa2f-a6dd90c6586a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:22:03.790944 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50195398-dfd7-4bde-aa2f-a6dd90c6586a admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:22:03.794634 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845ac47-6227-47c8-bfa0-9e24a7ce0a10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:22:03.795238 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845ac47-6227-47c8-bfa0-9e24a7ce0a10 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:03.796088 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845ac47-6227-47c8-bfa0-9e24a7ce0a10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:03.796377 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7845ac47-6227-47c8-bfa0-9e24a7ce0a10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:03.805099 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:22:03.809955 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7845ac47-6227-47c8-bfa0-9e24a7ce0a10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:22:03.812852 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7845ac47-6227-47c8-bfa0-9e24a7ce0a10 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:22:03.824806 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1285/2583] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:22:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:22:04.087177 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c611f2-feab-45cd-a2ae-11ed945fae4e None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:22:04.088625 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-c9c611f2-feab-45cd-a2ae-11ed945fae4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:22:04.103737 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c611f2-feab-45cd-a2ae-11ed945fae4e None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:22:04.108426 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb8e91-6758-4a74-b6df-1fb898e8e960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:22:04.109099 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb8e91-6758-4a74-b6df-1fb898e8e960 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:04.109378 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb8e91-6758-4a74-b6df-1fb898e8e960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:04.109656 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdb8e91-6758-4a74-b6df-1fb898e8e960 admin admin] PATH_INFO: `/v3/projects/c4e119d26eea436c9087e4eec13b221c` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:04.111933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdb8e91-6758-4a74-b6df-1fb898e8e960 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4e119d26eea436c9087e4eec13b221c)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:22:04.113859 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdb8e91-6758-4a74-b6df-1fb898e8e960 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:22:04.160966 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-1fdb8e91-6758-4a74-b6df-1fb898e8e960 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4e119d26eea436c9087e4eec13b221c', 'request_id': 'req-1fdb8e91-6758-4a74-b6df-1fb898e8e960'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:04.167544 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-1fdb8e91-6758-4a74-b6df-1fb898e8e960 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4e119d26eea436c9087e4eec13b221c', 'request_id': 'req-1fdb8e91-6758-4a74-b6df-1fb898e8e960'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:04.173323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-1fdb8e91-6758-4a74-b6df-1fb898e8e960 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4e119d26eea436c9087e4eec13b221c', 'request_id': 'req-1fdb8e91-6758-4a74-b6df-1fb898e8e960'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:04.184217 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-1fdb8e91-6758-4a74-b6df-1fb898e8e960 admin admin] The token cache is being invalidate because project c4e119d26eea436c9087e4eec13b221c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:22:04.184493 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-1fdb8e91-6758-4a74-b6df-1fb898e8e960 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:04.185125 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-1fdb8e91-6758-4a74-b6df-1fb898e8e960 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:04.186636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1299/2584] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:22:04 2022] DELETE /identity/v3/projects/c4e119d26eea436c9087e4eec13b221c => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:22:04.190635 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845ac47-6227-47c8-bfa0-9e24a7ce0a10 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:22:04.200280 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7845ac47-6227-47c8-bfa0-9e24a7ce0a10 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:22:04.257181 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec53ae7-997d-49e2-8b7f-03381a6e2e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:22:04.257809 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec53ae7-997d-49e2-8b7f-03381a6e2e6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:04.258090 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec53ae7-997d-49e2-8b7f-03381a6e2e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:04.258405 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec53ae7-997d-49e2-8b7f-03381a6e2e6a admin admin] PATH_INFO: `/v3/users/d23af9646efa478b951dc0081f99d539` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:04.259525 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec53ae7-997d-49e2-8b7f-03381a6e2e6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d23af9646efa478b951dc0081f99d539)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:22:04.261761 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec53ae7-997d-49e2-8b7f-03381a6e2e6a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:22:04.308442 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4ec53ae7-997d-49e2-8b7f-03381a6e2e6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd23af9646efa478b951dc0081f99d539'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:04.320576 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4ec53ae7-997d-49e2-8b7f-03381a6e2e6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd23af9646efa478b951dc0081f99d539'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:04.328016 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4ec53ae7-997d-49e2-8b7f-03381a6e2e6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd23af9646efa478b951dc0081f99d539'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:04.328715 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4ec53ae7-997d-49e2-8b7f-03381a6e2e6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd23af9646efa478b951dc0081f99d539'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:04.336174 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-4ec53ae7-997d-49e2-8b7f-03381a6e2e6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd23af9646efa478b951dc0081f99d539'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:04.342750 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1286/2585] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:22:04 2022] DELETE /identity/v3/users/d23af9646efa478b951dc0081f99d539 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:22:04.605087 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb8e91-6758-4a74-b6df-1fb898e8e960 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:22:04.606536 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-1fdb8e91-6758-4a74-b6df-1fb898e8e960 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:22:04.622489 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdb8e91-6758-4a74-b6df-1fb898e8e960 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:22:04.681231 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1377cc5c-0228-4e16-a8de-f4ed312ce7fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:22:04.681852 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1377cc5c-0228-4e16-a8de-f4ed312ce7fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:04.682128 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1377cc5c-0228-4e16-a8de-f4ed312ce7fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:04.682435 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1377cc5c-0228-4e16-a8de-f4ed312ce7fc admin admin] PATH_INFO: `/v3/projects/78a4bf75d8f34ed0a8ceb4616a351a66` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:04.683547 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1377cc5c-0228-4e16-a8de-f4ed312ce7fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=78a4bf75d8f34ed0a8ceb4616a351a66)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:22:04.685435 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1377cc5c-0228-4e16-a8de-f4ed312ce7fc admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:22:04.686936 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:22:04.733529 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-1377cc5c-0228-4e16-a8de-f4ed312ce7fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78a4bf75d8f34ed0a8ceb4616a351a66', 'request_id': 'req-1377cc5c-0228-4e16-a8de-f4ed312ce7fc'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:04.740606 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-1377cc5c-0228-4e16-a8de-f4ed312ce7fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78a4bf75d8f34ed0a8ceb4616a351a66', 'request_id': 'req-1377cc5c-0228-4e16-a8de-f4ed312ce7fc'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:04.746083 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-1377cc5c-0228-4e16-a8de-f4ed312ce7fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78a4bf75d8f34ed0a8ceb4616a351a66', 'request_id': 'req-1377cc5c-0228-4e16-a8de-f4ed312ce7fc'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:04.757402 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-1377cc5c-0228-4e16-a8de-f4ed312ce7fc admin admin] The token cache is being invalidate because project 78a4bf75d8f34ed0a8ceb4616a351a66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:22:04.757688 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-1377cc5c-0228-4e16-a8de-f4ed312ce7fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:04.758344 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-1377cc5c-0228-4e16-a8de-f4ed312ce7fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:04.759816 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1300/2586] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:22:04 2022] DELETE /identity/v3/projects/78a4bf75d8f34ed0a8ceb4616a351a66 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:22:04.763826 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec53ae7-997d-49e2-8b7f-03381a6e2e6a admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:22:04.765277 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-4ec53ae7-997d-49e2-8b7f-03381a6e2e6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:22:04.778376 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec53ae7-997d-49e2-8b7f-03381a6e2e6a admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:22:04.834158 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93d9dbc-efa4-47e3-bd3b-9afc1382a900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:22:04.834819 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d9dbc-efa4-47e3-bd3b-9afc1382a900 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:04.835098 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d9dbc-efa4-47e3-bd3b-9afc1382a900 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:04.835400 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93d9dbc-efa4-47e3-bd3b-9afc1382a900 admin admin] PATH_INFO: `/v3/users/eb3e4460a587473c88b388a3150f6849` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:04.836593 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93d9dbc-efa4-47e3-bd3b-9afc1382a900 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb3e4460a587473c88b388a3150f6849)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:22:04.838612 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93d9dbc-efa4-47e3-bd3b-9afc1382a900 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:22:04.852166 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:22:04.885706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f93d9dbc-efa4-47e3-bd3b-9afc1382a900 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb3e4460a587473c88b388a3150f6849'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:04.897759 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f93d9dbc-efa4-47e3-bd3b-9afc1382a900 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb3e4460a587473c88b388a3150f6849'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:04.906184 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f93d9dbc-efa4-47e3-bd3b-9afc1382a900 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb3e4460a587473c88b388a3150f6849'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:04.906807 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f93d9dbc-efa4-47e3-bd3b-9afc1382a900 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb3e4460a587473c88b388a3150f6849'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:04.913918 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-f93d9dbc-efa4-47e3-bd3b-9afc1382a900 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb3e4460a587473c88b388a3150f6849'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:04.919790 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1287/2587] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:22:04 2022] DELETE /identity/v3/users/eb3e4460a587473c88b388a3150f6849 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:22:05.189160 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1377cc5c-0228-4e16-a8de-f4ed312ce7fc admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:22:05.190725 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-1377cc5c-0228-4e16-a8de-f4ed312ce7fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:22:05.207100 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1377cc5c-0228-4e16-a8de-f4ed312ce7fc admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:22:05.267671 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ae97d5-e36f-4eae-8b14-e3a3edf9809d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:22:05.268306 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae97d5-e36f-4eae-8b14-e3a3edf9809d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:05.268589 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae97d5-e36f-4eae-8b14-e3a3edf9809d admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:05.268863 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ae97d5-e36f-4eae-8b14-e3a3edf9809d admin admin] PATH_INFO: `/v3/projects/59da7d8bd1df429588d9dd18743e01eb` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:05.269977 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae97d5-e36f-4eae-8b14-e3a3edf9809d admin admin] RBAC: Authorizing `identity:delete_project(project_id=59da7d8bd1df429588d9dd18743e01eb)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:22:05.271934 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ae97d5-e36f-4eae-8b14-e3a3edf9809d admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:22:05.317290 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-66ae97d5-e36f-4eae-8b14-e3a3edf9809d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59da7d8bd1df429588d9dd18743e01eb', 'request_id': 'req-66ae97d5-e36f-4eae-8b14-e3a3edf9809d'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:05.323808 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-66ae97d5-e36f-4eae-8b14-e3a3edf9809d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59da7d8bd1df429588d9dd18743e01eb', 'request_id': 'req-66ae97d5-e36f-4eae-8b14-e3a3edf9809d'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:05.329579 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-66ae97d5-e36f-4eae-8b14-e3a3edf9809d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59da7d8bd1df429588d9dd18743e01eb', 'request_id': 'req-66ae97d5-e36f-4eae-8b14-e3a3edf9809d'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:05.340834 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-66ae97d5-e36f-4eae-8b14-e3a3edf9809d admin admin] The token cache is being invalidate because project 59da7d8bd1df429588d9dd18743e01eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:22:05.341125 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-66ae97d5-e36f-4eae-8b14-e3a3edf9809d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:05.341760 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-66ae97d5-e36f-4eae-8b14-e3a3edf9809d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:05.343281 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1301/2588] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:22:05 2022] DELETE /identity/v3/projects/59da7d8bd1df429588d9dd18743e01eb => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:22:05.349640 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ce0e5-4bda-4035-a704-1a660bce80cf None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:05.349995 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ce0e5-4bda-4035-a704-1a660bce80cf None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:05.350390 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ce0e5-4bda-4035-a704-1a660bce80cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:05.370338 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-5b7ce0e5-4bda-4035-a704-1a660bce80cf None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:22:05.371487 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5b7ce0e5-4bda-4035-a704-1a660bce80cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:22:05.431002 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1288/2589] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:22:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:22:05.439295 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9fd7a-90d7-421a-9897-a6b4320c37d9 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:05.439586 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9fd7a-90d7-421a-9897-a6b4320c37d9 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:05.439863 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f9fd7a-90d7-421a-9897-a6b4320c37d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:05.459462 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-23f9fd7a-90d7-421a-9897-a6b4320c37d9 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:22:05.460697 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-23f9fd7a-90d7-421a-9897-a6b4320c37d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:22:05.468509 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1302/2590] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:22:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:22:05.473068 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ce0e5-4bda-4035-a704-1a660bce80cf None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:22:05.482843 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ce0e5-4bda-4035-a704-1a660bce80cf None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:22:05.488362 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0a06b-6ef4-407a-9696-2ca67d63f094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:22:05.488960 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0a06b-6ef4-407a-9696-2ca67d63f094 admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:05.489236 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0a06b-6ef4-407a-9696-2ca67d63f094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:05.489502 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f0a06b-6ef4-407a-9696-2ca67d63f094 admin admin] PATH_INFO: `/v3/domains` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:05.490168 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f0a06b-6ef4-407a-9696-2ca67d63f094 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81411) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 20:22:05.490502 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f0a06b-6ef4-407a-9696-2ca67d63f094 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:22:05.492070 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f0a06b-6ef4-407a-9696-2ca67d63f094 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:22:05.503700 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1289/2591] 10.222.0.13 () {58 vars in 1153 bytes} [Sun Oct 23 20:22:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 20:22:05.508020 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f9fd7a-90d7-421a-9897-a6b4320c37d9 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:22:05.517842 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f9fd7a-90d7-421a-9897-a6b4320c37d9 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:22:05.522523 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c22a9aa-1c3c-47f8-abc9-8b84dbfc1ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:22:05.523118 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22a9aa-1c3c-47f8-abc9-8b84dbfc1ea4 admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:05.523397 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22a9aa-1c3c-47f8-abc9-8b84dbfc1ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:05.523670 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c22a9aa-1c3c-47f8-abc9-8b84dbfc1ea4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:05.524309 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c22a9aa-1c3c-47f8-abc9-8b84dbfc1ea4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:22:05.526189 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c22a9aa-1c3c-47f8-abc9-8b84dbfc1ea4 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:22:05.538006 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:22:05.555221 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1303/2592] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:22:05 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 20:22:05.559281 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0a06b-6ef4-407a-9696-2ca67d63f094 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:22:05.569244 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f0a06b-6ef4-407a-9696-2ca67d63f094 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:22:05.628293 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf60a681-88e3-4957-8c26-4a21ef6f0e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:22:05.628822 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60a681-88e3-4957-8c26-4a21ef6f0e2e admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:05.629104 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60a681-88e3-4957-8c26-4a21ef6f0e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:05.629400 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60a681-88e3-4957-8c26-4a21ef6f0e2e admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:05.630020 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf60a681-88e3-4957-8c26-4a21ef6f0e2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:22:05.631969 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf60a681-88e3-4957-8c26-4a21ef6f0e2e admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:22:05.653277 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1290/2593] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:22:05 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:22:05.657492 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c22a9aa-1c3c-47f8-abc9-8b84dbfc1ea4 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:22:05.668175 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c22a9aa-1c3c-47f8-abc9-8b84dbfc1ea4 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:22:05.672657 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e18c18-8c91-4fd5-be2a-e12cb4137eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:22:05.673243 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e18c18-8c91-4fd5-be2a-e12cb4137eeb admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:05.673520 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e18c18-8c91-4fd5-be2a-e12cb4137eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:05.673845 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e18c18-8c91-4fd5-be2a-e12cb4137eeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:05.674518 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e18c18-8c91-4fd5-be2a-e12cb4137eeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:22:05.676134 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e18c18-8c91-4fd5-be2a-e12cb4137eeb admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:22:05.685745 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1304/2594] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:22:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:22:05.689445 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf60a681-88e3-4957-8c26-4a21ef6f0e2e admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:22:05.699184 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf60a681-88e3-4957-8c26-4a21ef6f0e2e admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:22:05.704072 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f708487-5ad3-4367-b9dc-aa8a1d42806a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:22:05.704663 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f708487-5ad3-4367-b9dc-aa8a1d42806a admin admin] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:05.705476 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f708487-5ad3-4367-b9dc-aa8a1d42806a admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:05.705775 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f708487-5ad3-4367-b9dc-aa8a1d42806a admin admin] PATH_INFO: `/v3/roles` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:05.706443 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f708487-5ad3-4367-b9dc-aa8a1d42806a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:22:05.708024 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f708487-5ad3-4367-b9dc-aa8a1d42806a admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:22:05.710026 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:22:05.719816 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1291/2595] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:22:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:22:05.724036 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e18c18-8c91-4fd5-be2a-e12cb4137eeb admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:22:05.733966 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e18c18-8c91-4fd5-be2a-e12cb4137eeb admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:22:05.738779 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef53ef45-2b63-418e-a8be-6de0097f2124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:22:05.739434 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53ef45-2b63-418e-a8be-6de0097f2124 admin admin] REQUEST_METHOD: `PUT` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:05.739718 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53ef45-2b63-418e-a8be-6de0097f2124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:05.740048 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53ef45-2b63-418e-a8be-6de0097f2124 admin admin] PATH_INFO: `/v3/projects/6c47edeb8166480cab6b8ecbe29ca50d/users/f76482be690149a680b40f996691c5fc/roles/69610ceae19c4a2b963433e2a9f5bb26` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:05.755655 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef53ef45-2b63-418e-a8be-6de0097f2124 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c47edeb8166480cab6b8ecbe29ca50d, user_id=f76482be690149a680b40f996691c5fc, role_id=69610ceae19c4a2b963433e2a9f5bb26)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:22:05.758327 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef53ef45-2b63-418e-a8be-6de0097f2124 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:22:05.771452 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1305/2596] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:22:05 2022] PUT /identity/v3/projects/6c47edeb8166480cab6b8ecbe29ca50d/users/f76482be690149a680b40f996691c5fc/roles/69610ceae19c4a2b963433e2a9f5bb26 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:22:05.784075 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e6a57-aef5-4cf4-bdc0-f06718e4a187 None None] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:05.784577 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e6a57-aef5-4cf4-bdc0-f06718e4a187 None None] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:05.784687 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3e6a57-aef5-4cf4-bdc0-f06718e4a187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:05.812163 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.auth.core [None req-9a3e6a57-aef5-4cf4-bdc0-f06718e4a187 None None] MFA Rules not processed for user `f76482be690149a680b40f996691c5fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81411) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:22:05.813359 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9a3e6a57-aef5-4cf4-bdc0-f06718e4a187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:22:05.877220 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1292/2597] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:22:05 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:22:05.881628 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef53ef45-2b63-418e-a8be-6de0097f2124 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:22:05.943268 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef53ef45-2b63-418e-a8be-6de0097f2124 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:22:05.947945 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42383fd5-8fc5-45b8-93b0-3fe937d1d09c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:22:05.948549 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42383fd5-8fc5-45b8-93b0-3fe937d1d09c admin admin] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:05.948828 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42383fd5-8fc5-45b8-93b0-3fe937d1d09c admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:05.949101 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42383fd5-8fc5-45b8-93b0-3fe937d1d09c admin admin] PATH_INFO: `/v3/projects` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:05.949735 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42383fd5-8fc5-45b8-93b0-3fe937d1d09c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:22:05.951916 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42383fd5-8fc5-45b8-93b0-3fe937d1d09c admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:22:05.978710 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1306/2598] 10.222.0.13 () {60 vars in 1153 bytes} [Sun Oct 23 20:22:05 2022] POST /identity/v3/projects => generated 385 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:22:05.982623 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3e6a57-aef5-4cf4-bdc0-f06718e4a187 None None] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:22:05.993134 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3e6a57-aef5-4cf4-bdc0-f06718e4a187 None None] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:22:06.049356 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f102a-4c4d-4bed-b113-598e02133eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:22:06.049980 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f102a-4c4d-4bed-b113-598e02133eaf admin admin] REQUEST_METHOD: `POST` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:06.050283 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f102a-4c4d-4bed-b113-598e02133eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:06.050562 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95f102a-4c4d-4bed-b113-598e02133eaf admin admin] PATH_INFO: `/v3/users` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:06.051203 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95f102a-4c4d-4bed-b113-598e02133eaf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:22:06.053124 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95f102a-4c4d-4bed-b113-598e02133eaf admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:22:06.076499 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1293/2599] 10.222.0.13 () {60 vars in 1144 bytes} [Sun Oct 23 20:22:05 2022] POST /identity/v3/users => generated 299 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 20:22:06.077690 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42383fd5-8fc5-45b8-93b0-3fe937d1d09c admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:22:06.088763 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42383fd5-8fc5-45b8-93b0-3fe937d1d09c admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:22:06.093403 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8756b73d-925f-4865-82e0-b577830a5a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:22:06.094003 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756b73d-925f-4865-82e0-b577830a5a55 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:06.094315 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756b73d-925f-4865-82e0-b577830a5a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:06.094592 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8756b73d-925f-4865-82e0-b577830a5a55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:06.095235 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8756b73d-925f-4865-82e0-b577830a5a55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:22:06.096854 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8756b73d-925f-4865-82e0-b577830a5a55 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:22:06.107256 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1307/2600] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:22:06.111259 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f102a-4c4d-4bed-b113-598e02133eaf admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:22:06.121032 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95f102a-4c4d-4bed-b113-598e02133eaf admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:22:06.125669 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e556e343-307b-40e1-9d4f-7c5d2d8019d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:22:06.126345 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556e343-307b-40e1-9d4f-7c5d2d8019d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:06.126631 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556e343-307b-40e1-9d4f-7c5d2d8019d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:06.127435 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e556e343-307b-40e1-9d4f-7c5d2d8019d9 admin admin] PATH_INFO: `/v3/projects/d40ec6ebccb64dd993ae43b4f9cb7137/users/76e28c1b647b46478e550aa676737e9f/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:06.142224 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e556e343-307b-40e1-9d4f-7c5d2d8019d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d40ec6ebccb64dd993ae43b4f9cb7137, user_id=76e28c1b647b46478e550aa676737e9f, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:22:06.145766 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e556e343-307b-40e1-9d4f-7c5d2d8019d9 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:22:06.158095 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1294/2601] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:22:06 2022] PUT /identity/v3/projects/d40ec6ebccb64dd993ae43b4f9cb7137/users/76e28c1b647b46478e550aa676737e9f/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:22:06.161993 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8756b73d-925f-4865-82e0-b577830a5a55 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:22:06.171659 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8756b73d-925f-4865-82e0-b577830a5a55 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:22:06.226617 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f750128b-0876-43a5-ac50-fb18de536b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:22:06.227208 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750128b-0876-43a5-ac50-fb18de536b42 admin admin] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:06.227510 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750128b-0876-43a5-ac50-fb18de536b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:06.227780 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f750128b-0876-43a5-ac50-fb18de536b42 admin admin] PATH_INFO: `/v3/roles` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:06.228412 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f750128b-0876-43a5-ac50-fb18de536b42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:22:06.229997 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f750128b-0876-43a5-ac50-fb18de536b42 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:22:06.239600 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1308/2602] 10.222.0.13 () {58 vars in 1122 bytes} [Sun Oct 23 20:22:06 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 20:22:06.243648 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e556e343-307b-40e1-9d4f-7c5d2d8019d9 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:22:06.254641 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e556e343-307b-40e1-9d4f-7c5d2d8019d9 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:22:06.259292 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bfcb27-9c25-4a3f-af74-ca59a009831b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:22:06.259945 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bfcb27-9c25-4a3f-af74-ca59a009831b admin admin] REQUEST_METHOD: `PUT` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:06.260230 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bfcb27-9c25-4a3f-af74-ca59a009831b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:06.260508 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bfcb27-9c25-4a3f-af74-ca59a009831b admin admin] PATH_INFO: `/v3/projects/d40ec6ebccb64dd993ae43b4f9cb7137/users/76e28c1b647b46478e550aa676737e9f/roles/742dfaf63ef74207a2c9213411513925` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:06.262484 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bfcb27-9c25-4a3f-af74-ca59a009831b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d40ec6ebccb64dd993ae43b4f9cb7137, user_id=76e28c1b647b46478e550aa676737e9f, role_id=742dfaf63ef74207a2c9213411513925)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:22:06.265132 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bfcb27-9c25-4a3f-af74-ca59a009831b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:22:06.275805 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1295/2603] 10.222.0.13 () {60 vars in 1483 bytes} [Sun Oct 23 20:22:06 2022] PUT /identity/v3/projects/d40ec6ebccb64dd993ae43b4f9cb7137/users/76e28c1b647b46478e550aa676737e9f/roles/742dfaf63ef74207a2c9213411513925 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:22:06.285737 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67562a1-9661-4589-9de9-d66130558879 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:06.286026 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67562a1-9661-4589-9de9-d66130558879 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:06.286357 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67562a1-9661-4589-9de9-d66130558879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:06.314122 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-d67562a1-9661-4589-9de9-d66130558879 None None] MFA Rules not processed for user `76e28c1b647b46478e550aa676737e9f`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:22:06.315964 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-d67562a1-9661-4589-9de9-d66130558879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:22:06.328437 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:22:06.384071 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1309/2604] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:22:06 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:22:06.395020 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bfcb27-9c25-4a3f-af74-ca59a009831b admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:22:06.396626 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-a8bfcb27-9c25-4a3f-af74-ca59a009831b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:22:06.440110 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:22:06.455153 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bfcb27-9c25-4a3f-af74-ca59a009831b admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:22:06.458796 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aea4f0-b433-4d35-b107-dce4666fcf30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:22:06.459431 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aea4f0-b433-4d35-b107-dce4666fcf30 service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:06.459712 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aea4f0-b433-4d35-b107-dce4666fcf30 service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:06.459989 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aea4f0-b433-4d35-b107-dce4666fcf30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:06.515719 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aea4f0-b433-4d35-b107-dce4666fcf30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:22:06.517143 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aea4f0-b433-4d35-b107-dce4666fcf30 service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:22:06.526006 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1296/2605] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:22:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:22:10.710991 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67562a1-9661-4589-9de9-d66130558879 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:22:10.712461 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-d67562a1-9661-4589-9de9-d66130558879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:22:10.724786 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67562a1-9661-4589-9de9-d66130558879 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:22:10.728373 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905971fd-b6f0-42e9-8a65-558ac56b0138 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:22:10.728968 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905971fd-b6f0-42e9-8a65-558ac56b0138 service cinder] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:10.729247 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905971fd-b6f0-42e9-8a65-558ac56b0138 service cinder] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:10.729518 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905971fd-b6f0-42e9-8a65-558ac56b0138 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:10.738941 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905971fd-b6f0-42e9-8a65-558ac56b0138 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:22:10.740380 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905971fd-b6f0-42e9-8a65-558ac56b0138 service cinder] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:22:10.749600 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1310/2606] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:22:10 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:22:15.629213 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aea4f0-b433-4d35-b107-dce4666fcf30 service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:22:15.690733 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aea4f0-b433-4d35-b107-dce4666fcf30 service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:22:15.695340 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160f154a-2ec4-4646-b34c-3fe8c749717b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:22:15.695986 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160f154a-2ec4-4646-b34c-3fe8c749717b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:15.696266 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160f154a-2ec4-4646-b34c-3fe8c749717b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:15.696540 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160f154a-2ec4-4646-b34c-3fe8c749717b admin admin] PATH_INFO: `/v3/users/f76482be690149a680b40f996691c5fc` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:15.697653 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160f154a-2ec4-4646-b34c-3fe8c749717b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f76482be690149a680b40f996691c5fc)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:22:15.699588 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160f154a-2ec4-4646-b34c-3fe8c749717b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:22:15.745326 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-160f154a-2ec4-4646-b34c-3fe8c749717b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f76482be690149a680b40f996691c5fc'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:15.757150 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-160f154a-2ec4-4646-b34c-3fe8c749717b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f76482be690149a680b40f996691c5fc'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:15.763626 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-160f154a-2ec4-4646-b34c-3fe8c749717b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f76482be690149a680b40f996691c5fc'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:15.764600 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-160f154a-2ec4-4646-b34c-3fe8c749717b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f76482be690149a680b40f996691c5fc'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:15.771584 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-160f154a-2ec4-4646-b34c-3fe8c749717b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f76482be690149a680b40f996691c5fc'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:15.777382 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1297/2607] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:22:15 2022] DELETE /identity/v3/users/f76482be690149a680b40f996691c5fc => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:22:15.781912 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44bddd-5bc0-4aec-b0d5-a941fd6ba036 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:15.782434 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44bddd-5bc0-4aec-b0d5-a941fd6ba036 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:15.782540 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d44bddd-5bc0-4aec-b0d5-a941fd6ba036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:15.802740 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-4d44bddd-5bc0-4aec-b0d5-a941fd6ba036 None None] MFA Rules not processed for user `55e5de29fcfb4f0abcb119685810b1da`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:22:15.803975 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-4d44bddd-5bc0-4aec-b0d5-a941fd6ba036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:22:15.861840 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1311/2608] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:22:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:22:15.872041 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160f154a-2ec4-4646-b34c-3fe8c749717b admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:22:15.873424 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-160f154a-2ec4-4646-b34c-3fe8c749717b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:22:15.887722 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:22:15.935236 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160f154a-2ec4-4646-b34c-3fe8c749717b admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:22:15.938844 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4710b69-b62a-4fb6-ac8f-302cc839638e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:22:15.939443 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4710b69-b62a-4fb6-ac8f-302cc839638e service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:15.939719 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4710b69-b62a-4fb6-ac8f-302cc839638e service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:15.939996 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4710b69-b62a-4fb6-ac8f-302cc839638e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:15.948787 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4710b69-b62a-4fb6-ac8f-302cc839638e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:22:15.950220 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4710b69-b62a-4fb6-ac8f-302cc839638e service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:22:15.959822 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1298/2609] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:22:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:22:16.220121 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d44bddd-5bc0-4aec-b0d5-a941fd6ba036 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:22:16.221643 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-4d44bddd-5bc0-4aec-b0d5-a941fd6ba036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:22:16.234264 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d44bddd-5bc0-4aec-b0d5-a941fd6ba036 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:22:16.239274 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd7d03-b2f5-42cb-8825-4ff14fa9d7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:22:16.240010 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd7d03-b2f5-42cb-8825-4ff14fa9d7da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:16.240288 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd7d03-b2f5-42cb-8825-4ff14fa9d7da admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:16.240555 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd7d03-b2f5-42cb-8825-4ff14fa9d7da admin admin] PATH_INFO: `/v3/projects/6c47edeb8166480cab6b8ecbe29ca50d` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:16.241811 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd7d03-b2f5-42cb-8825-4ff14fa9d7da admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c47edeb8166480cab6b8ecbe29ca50d)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:22:16.243855 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd7d03-b2f5-42cb-8825-4ff14fa9d7da admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:22:16.290746 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-0cdd7d03-b2f5-42cb-8825-4ff14fa9d7da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c47edeb8166480cab6b8ecbe29ca50d', 'request_id': 'req-0cdd7d03-b2f5-42cb-8825-4ff14fa9d7da'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:16.297752 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-0cdd7d03-b2f5-42cb-8825-4ff14fa9d7da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c47edeb8166480cab6b8ecbe29ca50d', 'request_id': 'req-0cdd7d03-b2f5-42cb-8825-4ff14fa9d7da'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:16.303518 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-0cdd7d03-b2f5-42cb-8825-4ff14fa9d7da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c47edeb8166480cab6b8ecbe29ca50d', 'request_id': 'req-0cdd7d03-b2f5-42cb-8825-4ff14fa9d7da'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:16.314397 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-0cdd7d03-b2f5-42cb-8825-4ff14fa9d7da admin admin] The token cache is being invalidate because project 6c47edeb8166480cab6b8ecbe29ca50d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:22:16.314688 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-0cdd7d03-b2f5-42cb-8825-4ff14fa9d7da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:16.315315 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-0cdd7d03-b2f5-42cb-8825-4ff14fa9d7da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:16.316817 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1312/2610] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:22:16 2022] DELETE /identity/v3/projects/6c47edeb8166480cab6b8ecbe29ca50d => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:22:16.320770 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4710b69-b62a-4fb6-ac8f-302cc839638e service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:22:16.330614 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4710b69-b62a-4fb6-ac8f-302cc839638e service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:22:16.386706 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae22d0e-d4f4-46d3-a006-9c8cbe513a9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:22:16.387330 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae22d0e-d4f4-46d3-a006-9c8cbe513a9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:16.387610 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae22d0e-d4f4-46d3-a006-9c8cbe513a9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:16.387882 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae22d0e-d4f4-46d3-a006-9c8cbe513a9b admin admin] PATH_INFO: `/v3/users/76e28c1b647b46478e550aa676737e9f` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:16.388989 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae22d0e-d4f4-46d3-a006-9c8cbe513a9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=76e28c1b647b46478e550aa676737e9f)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:22:16.390938 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae22d0e-d4f4-46d3-a006-9c8cbe513a9b admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:22:16.435438 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9ae22d0e-d4f4-46d3-a006-9c8cbe513a9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76e28c1b647b46478e550aa676737e9f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:16.448413 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9ae22d0e-d4f4-46d3-a006-9c8cbe513a9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76e28c1b647b46478e550aa676737e9f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:16.455829 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9ae22d0e-d4f4-46d3-a006-9c8cbe513a9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76e28c1b647b46478e550aa676737e9f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:16.456433 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9ae22d0e-d4f4-46d3-a006-9c8cbe513a9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76e28c1b647b46478e550aa676737e9f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:16.464014 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-9ae22d0e-d4f4-46d3-a006-9c8cbe513a9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76e28c1b647b46478e550aa676737e9f'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:16.469897 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1299/2611] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:22:16 2022] DELETE /identity/v3/users/76e28c1b647b46478e550aa676737e9f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:22:16.735312 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd7d03-b2f5-42cb-8825-4ff14fa9d7da admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:22:16.736741 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-0cdd7d03-b2f5-42cb-8825-4ff14fa9d7da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:22:16.752964 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd7d03-b2f5-42cb-8825-4ff14fa9d7da admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:22:16.773863 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:22:16.814740 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34359382-389f-404e-8e64-f20fb36f49b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:22:16.815356 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34359382-389f-404e-8e64-f20fb36f49b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:16.815576 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34359382-389f-404e-8e64-f20fb36f49b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:16.815850 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34359382-389f-404e-8e64-f20fb36f49b2 admin admin] PATH_INFO: `/v3/projects/d40ec6ebccb64dd993ae43b4f9cb7137` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:16.816985 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34359382-389f-404e-8e64-f20fb36f49b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d40ec6ebccb64dd993ae43b4f9cb7137)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:22:16.818921 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34359382-389f-404e-8e64-f20fb36f49b2 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:22:16.866795 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-34359382-389f-404e-8e64-f20fb36f49b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd40ec6ebccb64dd993ae43b4f9cb7137', 'request_id': 'req-34359382-389f-404e-8e64-f20fb36f49b2'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:16.873795 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-34359382-389f-404e-8e64-f20fb36f49b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd40ec6ebccb64dd993ae43b4f9cb7137', 'request_id': 'req-34359382-389f-404e-8e64-f20fb36f49b2'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:16.879667 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-34359382-389f-404e-8e64-f20fb36f49b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd40ec6ebccb64dd993ae43b4f9cb7137', 'request_id': 'req-34359382-389f-404e-8e64-f20fb36f49b2'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:16.890696 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-34359382-389f-404e-8e64-f20fb36f49b2 admin admin] The token cache is being invalidate because project d40ec6ebccb64dd993ae43b4f9cb7137 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81412) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:22:16.890981 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-34359382-389f-404e-8e64-f20fb36f49b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:16.891606 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-34359382-389f-404e-8e64-f20fb36f49b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:22:16.893067 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1313/2612] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:22:16 2022] DELETE /identity/v3/projects/d40ec6ebccb64dd993ae43b4f9cb7137 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:22:26.466467 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=73,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:22:26.887662 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=41,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:22:40.497696 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae22d0e-d4f4-46d3-a006-9c8cbe513a9b admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:22:40.499113 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-9ae22d0e-d4f4-46d3-a006-9c8cbe513a9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:22:40.554809 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae22d0e-d4f4-46d3-a006-9c8cbe513a9b admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:22:40.602156 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0a063b-36f1-43b0-a119-f6e4769ea7ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7bcc5f5101fb4913aaf5caf564ca6b60', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:22:40.602789 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a063b-36f1-43b0-a119-f6e4769ea7ce service cinder] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:40.603070 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a063b-36f1-43b0-a119-f6e4769ea7ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:40.603370 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a063b-36f1-43b0-a119-f6e4769ea7ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:40.605125 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3c0a063b-36f1-43b0-a119-f6e4769ea7ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:22:40.661946 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0a063b-36f1-43b0-a119-f6e4769ea7ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:22:40.663408 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0a063b-36f1-43b0-a119-f6e4769ea7ce service cinder] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:22:40.672163 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1300/2613] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:22:40 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:22:50.645677 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2690d-c0da-482c-994c-208f94f8d6e5 None None] REQUEST_METHOD: `POST` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:50.646169 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2690d-c0da-482c-994c-208f94f8d6e5 None None] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:50.646317 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e2690d-c0da-482c-994c-208f94f8d6e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:50.665497 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.auth.core [None req-a3e2690d-c0da-482c-994c-208f94f8d6e5 None None] MFA Rules not processed for user `2e15e1e499874c01b8d9b4aec9f9eb9c`. Rule list: `[]` (Enabled: `True`). {{(pid=81412) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 20:22:50.666749 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-a3e2690d-c0da-482c-994c-208f94f8d6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:22:50.668073 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=68 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:22:50.673929 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1314/2614] 10.222.0.13 () {58 vars in 958 bytes} [Sun Oct 23 20:22:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 20:22:50.994816 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0a063b-36f1-43b0-a119-f6e4769ea7ce service cinder] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:22:50.996213 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-3c0a063b-36f1-43b0-a119-f6e4769ea7ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:22:51.052486 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0a063b-36f1-43b0-a119-f6e4769ea7ce service cinder] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:22:51.056170 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f38bba3-7707-4e18-bd4f-6a82952e0c9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:22:51.056713 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38bba3-7707-4e18-bd4f-6a82952e0c9d service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:51.056994 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38bba3-7707-4e18-bd4f-6a82952e0c9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:51.057290 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f38bba3-7707-4e18-bd4f-6a82952e0c9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:51.059007 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5f38bba3-7707-4e18-bd4f-6a82952e0c9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:22:51.068880 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f38bba3-7707-4e18-bd4f-6a82952e0c9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:22:51.070312 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f38bba3-7707-4e18-bd4f-6a82952e0c9d service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:22:51.078508 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1301/2615] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:22:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:22:51.518331 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2690d-c0da-482c-994c-208f94f8d6e5 None None] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:22:51.519744 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-a3e2690d-c0da-482c-994c-208f94f8d6e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:22:51.591898 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e2690d-c0da-482c-994c-208f94f8d6e5 None None] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:22:51.596928 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35fe5fc-7e54-4626-acd7-20a58055fedb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312489c49a304ca9aacebd9843a4b65f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:22:51.597538 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35fe5fc-7e54-4626-acd7-20a58055fedb service placement] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:51.597846 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35fe5fc-7e54-4626-acd7-20a58055fedb service placement] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:51.598710 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35fe5fc-7e54-4626-acd7-20a58055fedb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:51.600437 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-d35fe5fc-7e54-4626-acd7-20a58055fedb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:22:51.611591 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35fe5fc-7e54-4626-acd7-20a58055fedb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:22:51.613145 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35fe5fc-7e54-4626-acd7-20a58055fedb service placement] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:22:51.623467 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1315/2616] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:22:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:22:54.161457 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f38bba3-7707-4e18-bd4f-6a82952e0c9d service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:22:54.165604 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5f38bba3-7707-4e18-bd4f-6a82952e0c9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:22:54.251957 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f38bba3-7707-4e18-bd4f-6a82952e0c9d service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:22:54.257478 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9381586-7b5e-4a74-930a-484b30fb10e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:22:54.258088 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9381586-7b5e-4a74-930a-484b30fb10e9 service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:22:54.259012 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9381586-7b5e-4a74-930a-484b30fb10e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:22:54.259297 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9381586-7b5e-4a74-930a-484b30fb10e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:22:54.261028 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f9381586-7b5e-4a74-930a-484b30fb10e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:22:54.277148 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9381586-7b5e-4a74-930a-484b30fb10e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:22:54.279338 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9381586-7b5e-4a74-930a-484b30fb10e9 service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:22:54.291160 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1302/2617] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:22:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 20:23:01.619279 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=35 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:23:04.285881 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=60 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:23:05.962674 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35fe5fc-7e54-4626-acd7-20a58055fedb service placement] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:23:05.976556 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35fe5fc-7e54-4626-acd7-20a58055fedb service placement] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:23:05.982331 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9377104e-0bba-44bb-b309-80f9864072fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:23:05.982947 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377104e-0bba-44bb-b309-80f9864072fc service nova] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:23:05.983227 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377104e-0bba-44bb-b309-80f9864072fc service nova] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:23:05.983498 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9377104e-0bba-44bb-b309-80f9864072fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:23:05.985252 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-9377104e-0bba-44bb-b309-80f9864072fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:23:05.998176 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9377104e-0bba-44bb-b309-80f9864072fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:23:05.999859 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9377104e-0bba-44bb-b309-80f9864072fc service nova] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:23:06.012059 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1316/2618] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:23:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:23:09.283206 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9381586-7b5e-4a74-930a-484b30fb10e9 service nova] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:23:09.284958 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-f9381586-7b5e-4a74-930a-484b30fb10e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:23:09.296206 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9381586-7b5e-4a74-930a-484b30fb10e9 service nova] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:23:09.299797 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4372308-1c35-4537-b3d7-82346749c046 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:23:09.300391 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4372308-1c35-4537-b3d7-82346749c046 service neutron] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:23:09.300668 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4372308-1c35-4537-b3d7-82346749c046 service neutron] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:23:09.301450 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4372308-1c35-4537-b3d7-82346749c046 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:23:09.303138 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d4372308-1c35-4537-b3d7-82346749c046 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:23:09.365055 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4372308-1c35-4537-b3d7-82346749c046 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:23:09.366499 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4372308-1c35-4537-b3d7-82346749c046 service neutron] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:23:09.375867 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1303/2619] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:23:09 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:23:16.006569 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=6 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:23:19.371625 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=30 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:23:33.625933 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9377104e-0bba-44bb-b309-80f9864072fc service nova] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:23:33.627488 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-9377104e-0bba-44bb-b309-80f9864072fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:23:33.711957 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9377104e-0bba-44bb-b309-80f9864072fc service nova] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:23:33.717152 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edf46cb-35e5-4cbb-b953-96f0b729a7b0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312489c49a304ca9aacebd9843a4b65f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:23:33.717749 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf46cb-35e5-4cbb-b953-96f0b729a7b0 service placement] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:23:33.718030 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf46cb-35e5-4cbb-b953-96f0b729a7b0 service placement] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:23:33.718333 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edf46cb-35e5-4cbb-b953-96f0b729a7b0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:23:33.720034 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-9edf46cb-35e5-4cbb-b953-96f0b729a7b0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:23:33.731212 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edf46cb-35e5-4cbb-b953-96f0b729a7b0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:23:33.732781 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edf46cb-35e5-4cbb-b953-96f0b729a7b0 service placement] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:23:33.742636 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1317/2620] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:23:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:23:43.738321 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=37 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:25:20.412868 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4372308-1c35-4537-b3d7-82346749c046 service neutron] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:25:20.414446 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-d4372308-1c35-4537-b3d7-82346749c046 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:25:20.510340 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4372308-1c35-4537-b3d7-82346749c046 service neutron] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:25:20.517271 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e12ff-11e4-4134-9631-f126bb71abfb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0b3d512304254ffcaa8dad90abf1f9c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:25:20.518405 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e12ff-11e4-4134-9631-f126bb71abfb service nova] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:25:20.518741 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e12ff-11e4-4134-9631-f126bb71abfb service nova] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:25:20.518965 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e12ff-11e4-4134-9631-f126bb71abfb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:25:20.520800 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-8c1e12ff-11e4-4134-9631-f126bb71abfb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:25:20.539390 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e12ff-11e4-4134-9631-f126bb71abfb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:25:20.541603 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e12ff-11e4-4134-9631-f126bb71abfb service nova] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:25:20.558650 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1304/2621] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:25:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:25:25.877001 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edf46cb-35e5-4cbb-b953-96f0b729a7b0 service placement] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:25:25.896768 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edf46cb-35e5-4cbb-b953-96f0b729a7b0 service placement] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:25:25.900405 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949114d-0b97-4f37-b529-b0d408e97bb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba8ff3413bc549d29486d1c724a1f383', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:25:25.901007 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949114d-0b97-4f37-b529-b0d408e97bb6 service neutron] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:25:25.901287 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949114d-0b97-4f37-b529-b0d408e97bb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:25:25.901559 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8949114d-0b97-4f37-b529-b0d408e97bb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:25:25.903326 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-8949114d-0b97-4f37-b529-b0d408e97bb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:25:25.986413 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8949114d-0b97-4f37-b529-b0d408e97bb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:25:25.987830 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8949114d-0b97-4f37-b529-b0d408e97bb6 service neutron] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:25:26.006557 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1318/2622] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:25:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:25:29.041218 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e12ff-11e4-4134-9631-f126bb71abfb service nova] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:25:29.054492 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e12ff-11e4-4134-9631-f126bb71abfb service nova] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:25:29.061526 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f15c3-7485-43dc-b358-598c71be28bc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312489c49a304ca9aacebd9843a4b65f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:25:29.062580 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f15c3-7485-43dc-b358-598c71be28bc service placement] REQUEST_METHOD: `GET` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:25:29.063152 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f15c3-7485-43dc-b358-598c71be28bc service placement] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:25:29.063661 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08f15c3-7485-43dc-b358-598c71be28bc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:25:29.065698 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-e08f15c3-7485-43dc-b358-598c71be28bc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:25:29.079539 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f15c3-7485-43dc-b358-598c71be28bc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:25:29.081211 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08f15c3-7485-43dc-b358-598c71be28bc service placement] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:25:29.093177 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1305/2623] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:25:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:25:32.250592 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949114d-0b97-4f37-b529-b0d408e97bb6 service neutron] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:25:32.252106 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-8949114d-0b97-4f37-b529-b0d408e97bb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:25:32.265148 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8949114d-0b97-4f37-b529-b0d408e97bb6 service neutron] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:25:32.270424 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799c292-bd39-4d98-8948-03c26428d399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:25:32.271050 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799c292-bd39-4d98-8948-03c26428d399 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:25:32.271330 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799c292-bd39-4d98-8948-03c26428d399 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:25:32.271603 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f799c292-bd39-4d98-8948-03c26428d399 admin admin] PATH_INFO: `/v3/users/2e15e1e499874c01b8d9b4aec9f9eb9c` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:25:32.272730 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f799c292-bd39-4d98-8948-03c26428d399 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e15e1e499874c01b8d9b4aec9f9eb9c)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:25:32.274699 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f799c292-bd39-4d98-8948-03c26428d399 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:25:32.320455 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-f799c292-bd39-4d98-8948-03c26428d399 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e15e1e499874c01b8d9b4aec9f9eb9c'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:25:32.333060 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-f799c292-bd39-4d98-8948-03c26428d399 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e15e1e499874c01b8d9b4aec9f9eb9c'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:25:32.339992 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-f799c292-bd39-4d98-8948-03c26428d399 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e15e1e499874c01b8d9b4aec9f9eb9c'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:25:32.347364 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-f799c292-bd39-4d98-8948-03c26428d399 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e15e1e499874c01b8d9b4aec9f9eb9c'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:25:32.348059 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-f799c292-bd39-4d98-8948-03c26428d399 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e15e1e499874c01b8d9b4aec9f9eb9c'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:25:32.353860 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1319/2624] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:25:32 2022] DELETE /identity/v3/users/2e15e1e499874c01b8d9b4aec9f9eb9c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:25:32.527208 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f15c3-7485-43dc-b358-598c71be28bc service placement] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:25:32.539386 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08f15c3-7485-43dc-b358-598c71be28bc service placement] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:25:32.596520 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b02ae90-651e-44db-8175-0c2a9b2c9714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:25:32.597078 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b02ae90-651e-44db-8175-0c2a9b2c9714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:25:32.597355 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b02ae90-651e-44db-8175-0c2a9b2c9714 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:25:32.597628 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b02ae90-651e-44db-8175-0c2a9b2c9714 admin admin] PATH_INFO: `/v3/projects/deacebf04e2d4a8093425ce7d16ca3cf` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:25:32.598759 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b02ae90-651e-44db-8175-0c2a9b2c9714 admin admin] RBAC: Authorizing `identity:delete_project(project_id=deacebf04e2d4a8093425ce7d16ca3cf)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:25:32.600638 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b02ae90-651e-44db-8175-0c2a9b2c9714 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:25:32.649672 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5b02ae90-651e-44db-8175-0c2a9b2c9714 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'deacebf04e2d4a8093425ce7d16ca3cf', 'request_id': 'req-5b02ae90-651e-44db-8175-0c2a9b2c9714'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:25:32.656549 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5b02ae90-651e-44db-8175-0c2a9b2c9714 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'deacebf04e2d4a8093425ce7d16ca3cf', 'request_id': 'req-5b02ae90-651e-44db-8175-0c2a9b2c9714'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:25:32.662265 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5b02ae90-651e-44db-8175-0c2a9b2c9714 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'deacebf04e2d4a8093425ce7d16ca3cf', 'request_id': 'req-5b02ae90-651e-44db-8175-0c2a9b2c9714'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:25:32.669260 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:25:32.674514 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5b02ae90-651e-44db-8175-0c2a9b2c9714 admin admin] The token cache is being invalidate because project deacebf04e2d4a8093425ce7d16ca3cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:25:32.674976 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5b02ae90-651e-44db-8175-0c2a9b2c9714 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:25:32.676538 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-5b02ae90-651e-44db-8175-0c2a9b2c9714 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:25:32.678129 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1306/2625] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:25:32 2022] DELETE /identity/v3/projects/deacebf04e2d4a8093425ce7d16ca3cf => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:25:32.682103 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799c292-bd39-4d98-8948-03c26428d399 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:25:32.683538 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-f799c292-bd39-4d98-8948-03c26428d399 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:25:32.696971 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f799c292-bd39-4d98-8948-03c26428d399 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:25:32.723382 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:25:32.755323 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d45d49a-3bf4-40f1-97ba-0ff89ed53057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:25:32.755944 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45d49a-3bf4-40f1-97ba-0ff89ed53057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:25:32.756218 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45d49a-3bf4-40f1-97ba-0ff89ed53057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:25:32.756484 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d45d49a-3bf4-40f1-97ba-0ff89ed53057 admin admin] PATH_INFO: `/v3/users/958f84e23a744656b292d68702721ed5` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:25:32.757600 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d45d49a-3bf4-40f1-97ba-0ff89ed53057 admin admin] RBAC: Authorizing `identity:delete_user(user_id=958f84e23a744656b292d68702721ed5)` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:25:32.759520 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d45d49a-3bf4-40f1-97ba-0ff89ed53057 admin admin] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:25:32.805926 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-3d45d49a-3bf4-40f1-97ba-0ff89ed53057 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '958f84e23a744656b292d68702721ed5'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:25:32.817643 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-3d45d49a-3bf4-40f1-97ba-0ff89ed53057 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '958f84e23a744656b292d68702721ed5'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:25:32.824449 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-3d45d49a-3bf4-40f1-97ba-0ff89ed53057 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '958f84e23a744656b292d68702721ed5'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:25:32.831268 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-3d45d49a-3bf4-40f1-97ba-0ff89ed53057 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '958f84e23a744656b292d68702721ed5'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:25:32.832029 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.notifications [None req-3d45d49a-3bf4-40f1-97ba-0ff89ed53057 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '958f84e23a744656b292d68702721ed5'} {{(pid=81412) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:25:32.837879 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1320/2626] 10.222.0.13 () {58 vars in 1224 bytes} [Sun Oct 23 20:25:32 2022] DELETE /identity/v3/users/958f84e23a744656b292d68702721ed5 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 20:25:33.010890 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b02ae90-651e-44db-8175-0c2a9b2c9714 admin admin] Authenticating user token {{(pid=81411) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:25:33.012363 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.fernet_utils [None req-5b02ae90-651e-44db-8175-0c2a9b2c9714 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81411) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:25:33.028699 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b02ae90-651e-44db-8175-0c2a9b2c9714 admin admin] Validating token access rules against request {{(pid=81411) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:25:33.092876 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eaa02db-2942-4589-b6a1-824786fc4e01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '55e5de29fcfb4f0abcb119685810b1da', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c6d0fd21d5e64f76b7baefae60befbed', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81411) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:25:33.093515 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa02db-2942-4589-b6a1-824786fc4e01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:25:33.093819 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa02db-2942-4589-b6a1-824786fc4e01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:25:33.094096 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eaa02db-2942-4589-b6a1-824786fc4e01 admin admin] PATH_INFO: `/v3/projects/1b7ac947dae84984adc4c779cb78e4c3` {{(pid=81411) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:25:33.095240 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaa02db-2942-4589-b6a1-824786fc4e01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b7ac947dae84984adc4c779cb78e4c3)` {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:25:33.097691 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eaa02db-2942-4589-b6a1-824786fc4e01 admin admin] RBAC: Authorization granted {{(pid=81411) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:25:33.142616 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-1eaa02db-2942-4589-b6a1-824786fc4e01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b7ac947dae84984adc4c779cb78e4c3', 'request_id': 'req-1eaa02db-2942-4589-b6a1-824786fc4e01'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:25:33.150113 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-1eaa02db-2942-4589-b6a1-824786fc4e01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b7ac947dae84984adc4c779cb78e4c3', 'request_id': 'req-1eaa02db-2942-4589-b6a1-824786fc4e01'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:25:33.155709 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-1eaa02db-2942-4589-b6a1-824786fc4e01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b7ac947dae84984adc4c779cb78e4c3', 'request_id': 'req-1eaa02db-2942-4589-b6a1-824786fc4e01'} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:25:33.167220 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-1eaa02db-2942-4589-b6a1-824786fc4e01 admin admin] The token cache is being invalidate because project 1b7ac947dae84984adc4c779cb78e4c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81411) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 20:25:33.167510 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-1eaa02db-2942-4589-b6a1-824786fc4e01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:25:33.168158 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG keystone.notifications [None req-1eaa02db-2942-4589-b6a1-824786fc4e01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81411) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 20:25:33.169617 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: [pid: 81411|app: 0|req: 1307/2627] 10.222.0.13 () {58 vars in 1233 bytes} [Sun Oct 23 20:25:33 2022] DELETE /identity/v3/projects/1b7ac947dae84984adc4c779cb78e4c3 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 20:25:42.834525 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=40,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:25:43.164297 dv-cinder-862432-1-iscsi devstack@keystone.service[81411]: DEBUG dbcounter [-] [81411] Writing DB stats keystone:SELECT=52,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81411) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 20:28:37.584122 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d45d49a-3bf4-40f1-97ba-0ff89ed53057 admin admin] Authenticating user token {{(pid=81412) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 20:28:37.585526 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-3d45d49a-3bf4-40f1-97ba-0ff89ed53057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:28:37.654545 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d45d49a-3bf4-40f1-97ba-0ff89ed53057 admin admin] Validating token access rules against request {{(pid=81412) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 20:28:37.712136 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5ba9e8-5236-4aa3-b86a-7c3e68b50364 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '312489c49a304ca9aacebd9843a4b65f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd40947427f5f43e7b6c82e3605737767', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81412) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 20:28:37.712748 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ba9e8-5236-4aa3-b86a-7c3e68b50364 service placement] REQUEST_METHOD: `GET` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 20:28:37.713027 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ba9e8-5236-4aa3-b86a-7c3e68b50364 service placement] SCRIPT_NAME: `/identity` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 20:28:37.713298 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5ba9e8-5236-4aa3-b86a-7c3e68b50364 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81412) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 20:28:37.715064 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.fernet_utils [None req-de5ba9e8-5236-4aa3-b86a-7c3e68b50364 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81412) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 20:28:37.726268 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ba9e8-5236-4aa3-b86a-7c3e68b50364 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 20:28:37.727840 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5ba9e8-5236-4aa3-b86a-7c3e68b50364 service placement] RBAC: Authorization granted {{(pid=81412) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 20:28:37.737759 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: [pid: 81412|app: 0|req: 1321/2628] 10.222.0.13 () {60 vars in 1365 bytes} [Sun Oct 23 20:28:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 20:28:47.733468 dv-cinder-862432-1-iscsi devstack@keystone.service[81412]: DEBUG dbcounter [-] [81412] Writing DB stats keystone:SELECT=58 {{(pid=81412) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}